Dr. John Ioannidis is a meta-researcher, and in this piece at The Atlantic, he explains how bias and a lack of true understanding of randomness renders many of our medical studies equivocal. Researchers go in to a study looking for a correlation, and viola! they find it. Sometimes, another team goes in looking for the opposite correlation, and viola! they find it. Frequently, however, the hallmark of science: That another team will attempt to duplicate your study and either add to the evidence or highlight an error - never occurs.
Naturally, drug studies are the worst. Naturally, because most of us intuitively grasp that where big money is involved, where a great deal is at stake, the research can easily be corrupted. It's corrupted in this sense because the researchers want to see the correlations between their new drug and the benefits - and that inherent bias leads them to set up a study that will naturally prove, rather than disprove the efficacy of their new product.
I've seen the type of bias Dr. Ionnidis is capturing first hand, amongst myself and my peers. It's much easier to set up a test or study that duplicates our beliefs about how something works; it is much, much harder to devise a test that could actually disprove (and hence validate if passed) our belief in how a system works.
Correlation and causation continue to be difficult to tease out of the world around us - we don't know much, and Dr. Ionnidis research has cast doubt on much of what we thought we knew. Read the article: It is fascinating!
I think my next project will be to download the study mentioned and read about the 45 pivotal studies Dr. Ionnidis concentrated on - what medical and nutrition advice that we take for granted is uncertain?
(If you can't follow the links, here is the complete url:
http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/
)
No comments:
Post a Comment