A meta-analysis of the health benefits of organic foods was published in The Annals of Internal Medicine on September 4, 2012. A large team of physicians and graduate students affiliated with the prestigious Stanford Medical School carried out the work. The paper’s basic finding was that consuming organic food does not deliver “clinically significant” improvements in human health. The Stanford press office and several team members communicated the study’s findings in a way that was designed to attract the media and stimulate debate. They obviously accomplished both goals.
The global media picked up and covered the paper’s “surprising” findings, which were often accompanied by headlines or commentary asserting, without qualification, that “…organic food delivers no meaningful health benefits…,” or variations on this theme. Media coverage of the study set off the latest round of discussion and spirited debate about whether organic food delivers on its multiple human health-related promises.
A small handful of scientists have carefully read, critiqued and summarized the results of the approximately 250 valid studies comparing the nutrient content and safety of organic versus conventional foods. Sophisticated methods have been established to conduct meta-analyses in ways that control for the two dozen or more factors that can influence a given study’s outcomes, in addition to the selection of the farming system (e.g., “conventional” versus “organic”). Many published meta-analyses by experts in this field have reached strikingly different conclusions from this same body of literature.
Why? Some people might wonder whether scientists really know how to measure food’s nutritional quality and safety. Others are losing faith in science because so much of it appears to be agenda-driven fodder for PR and political campaigns, and responsive to the highest bidder. As public support for science has dwindled, public interest in science has too often been eroded, or just buried, in the sheer volume of other science “messaging.”
The Fundamental Problem with the Stanford Study
The many methodological and technical flaws in the Stanford study are laid out clearly in the “Letters to the Editor” that are now posted on the website of The Annals of Internal Medicine. The methodological flaws are serious and because of them, the Stanford study will likely be remembered more for the dust it stirred up than the new light it sheds. But perhaps the most important problem with the Stanford study has remained, for the most part, below the radar.
The Stanford team looked for evidence in published studies of a “clinically significant” improvement in human health associated with consumption of organic food. This decision virtually guaranteed the outcome and also set up a straw man that critics of organic food will take pleasure in torching over and over.
To a physician, a “clinically significant difference” occurs when a patient comes into the doctor’s office with X problem, which is diagnosed, and the physician then recommends a course of action. It might include medication, a procedure and/or lifestyle changes. The physician draws on evidence from highly refined, often long-term clinical trials that establish a reliable linkage between a defined treatment or course of action and a clinically significant improvement in the patient’s health status.
The key characteristic embedded in a “clinically significant difference” is that the doctor must believe that for this specific patient, the person will get better if they do x, y and z, as prescribed by the doctor. If the person does not, the physician knows that he or she will face various consequences, ranging from a sicker and disappointed (or angry) patient to a call from the patient’s lawyer.
In nearly all cases, the decision to buy organic food, and the health impacts stemming from that decision, will not rise to the level of delivering – with a high level of reliability – a clinically significant improvement in health outcomes for a person dealing with an acute or chronic illness. We don’t need a meta-analysis to prove this. But does this mean there are no health benefits from consumption of organic food? No, it does not, because so many factors contribute to the etiology of disease. In fact, relatively little is known about what makes one person healthy and another ill. The cause of most cancers remains a mystery, 70 percent of birth defects remain of unknown cause, and even for many common diseases like diabetes, physicians cannot answer the patient’s first and most fundamental question – why did this happen to me?
By reducing a number of known risk factors for any one of dozens of health problems, consumption of organic food tips the trajectory of population-wide health outcomes in a more health-promoting direction. By how much, no one knows. Long-term clinical trials involving consumption of organic food have never been done and would be very, very expensive.
Consuming organic food does not guarantee any one individual dealing with a health problem a clinically significant improvement in health, because too many other variables and risk factors are in play with each individual. No one knows which risk factor might come into play, worsening an existing problem or creating a new one. The Stanford team ignored this critical distinction between health promotion/disease prevention and bringing about clinically significant improvement in the health of a sick person. Whether this was an act of omission or a poorly executed effort to erode consumer confidence in organic food, everyone who cares about promoting food’s nutritional quality and safety through sound science should expect more from a great medical school and a highly respected medical journal.
Dr. Benbrook’s full analysis of the Stanford study can be found here.© Food Safety News