Saturday, April 30, 2011
brain volume hersenen cognition
April 26, 2011 — Many studies have linked brain volume abnormalities to a variety of mental health conditions, including major depressive disorder (MDD), bipolar disorder, and schizophrenia.
But the author of an analysis published online April 4 in Archives of General Psychiatry concludes that the number of statistically significant results in the brain volume literature is "way too large to be true."
 |
| Dr. John P. A. Ioannidis |
"This pattern suggests strong biases in the literature," writes John P. A. Ioannidis, MD, DSc, of Stanford University's Prevention Research Center in California. "Selective outcome reporting and selective analyses reporting" are 2 possible explanations for the "excess significance bias" uncovered in the literature on brain volume abnormalities.
Reached for comment on the analysis, John D. Port, MD, PhD, associate professor of radiology and assistant professor of psychiatry at Mayo Clinic, Rochester, Minnesota, called Dr. Ioannidis' article and analysis "pretty darn good and not surprising at all." The excess of statistically significant results, Dr. Port said, "is a side effect of statistics and publication bias, in my opinion."
Many Links 'Likely Spurious'
Dr. Ioannidis has published numerous papers on a variety of medical topics that question the methods and interpretations of clinical trials and the ideas they generate. One of his most widely read publication may be the 2005 essay published in PLoS Medicine entitled "Why Most Published Research Findings Are False."
In his latest analysis, he used an "excess significance test" to evaluate whether there are too many reported studies in the brain volume literature that have statistically significant results.
"In a nutshell...there are too many studies done in the field showing too many significant results with brain volume abnormalities [and] many of these significant associations are likely to be spurious," Dr. Ioannidis told Medscape Medical News.
From 8 articles, he evaluated 41 meta-analyses with 461 data sets pertaining to brain volume abnormalities in 7 conditions: major depressive disorder, bipolar disorder, obsessive-compulsive disorder, posttraumatic stress disorder, autism, first-episode schizophrenia, and relatives of patients with schizophrenia.
Of the 41 meta-anlayses, roughly half (n = 21) found "statistically significant" associations, and 142 of the 461 data sets (31%) found a "positive" association.
"Even if the effect sizes observed in the meta-analyses are accurate, the number of positive results (n = 142) is almost double than what would have been expected (n = 78) based on power calculations for the included samples," said Dr. Iaonnidis.
And no condition is spared. On the basis of his research, "bias may be present in meta-analyses of all 7 examined conditions and in most of the examined brain structures," Dr. Ioannidis writes.
"The whole field needs transparent design and reporting of its results and careful reappraisal of putative associations," he said.
'Statistical Noise'
Dr. Port, who was not involved in the analysis, said scientists "need to supply the full data set (positive and negative analyses) and publishers need to be willing to publish the full data set so meta-analyses have the full data to look at."
Publication bias toward the positive findings, he added, "creates, in my opinion, what I like to call statistical noise.
"We know if you set a P value of .05, 1 out of every 20 results is going to be abnormally positive, by definition. So if you are only going to publish positive results, you're going to be publishing some by mistake [noise results], and these show up in these meta-analyses as a bias toward excess significance," Dr. Port explained.
To tackle this issue, "we can start by using more rigorous statistical limits, so instead of using a P value of .05 — use .01 — which means a lot less stuff will be significant, which, unfortunately, means a lot less papers will be published," he added.
The Bonferroni correction can also help. "We hate to do Bonferroni correction because you take a whole bunch of positive results and you get rid of them, but it's a very rigorous test, and if something is really positive, it will survive a Bonferroni correction."
The analysis had no funding. Dr. Ioannidis and Dr. Port have disclosed no relevant financial relationships.
Arch Gen Psychiatry. Published online April 4, 2011.
