Why You Should Be Skeptical of the Latest Nutrition Headlines: Part 2

This article is Part 2 of a two-part series about the problems with nutrition research and the way it’s presented in the media. For more reasons why you should be skeptical of the latest nutrition headlines, check out Part 1 of this series.

In my last article in this series, I talked about why observational studies aren’t a great tool for proving causal relationships; how the data collection methods researchers use rely on memory, not facts; how the healthy-user bias can impact study results; and how, in many cases, nutritional studies uncover “risks” that look an awful lot like pure chance. In this post, I’ll delve deeper into the reasons why you should take nutrition headlines with a grain of salt.

Some Scientific Results Can’t Be Replicated

Science works by experiments that can be repeated; when they are repeated, they must give the same answer. If an experiment does not replicate, something has gone wrong. – Young & Karr, The Royal Statistical Society (1)

As Young and Karr suggest above, replication is a key feature of the scientific method. An initial finding does not carry much weight on its own. For it to be considered valid, it needs to be replicated by other researchers.

We’re supposed to trust nutrition researchers to help us understand our health, but in some cases, the way they think about nutrition is faulty. Check out more reasons why you should remain skeptical of nutrition headlines.

 

In the context of nutrition research, because observational studies cannot prove causality, their findings should ideally be replicated in a randomized controlled trial
(RCT). RCTs are specifically designed to prove causality, and while not perfect (see below), they are much more persuasive as evidence than observational studies.

The results from most observational nutrition studies have not been replicated by RCTs. In fact, one analysis found that:

Zero of 52 nutrition claims from observational studies for a wide variety of dietary patterns and nutrient supplementation were replicated, and five claims were statistically significant in the opposite direction.

Yes, you read that correctly. Out of 52 claims made in observational nutrition studies, zero were replicated and five indicated the opposite of what the observational study suggested!

Let’s look at a specific example. Observational studies suggested that people with the highest intakes of beta-carotene, an antioxidant nutrient found primarily in fruits and vegetables, had a 31 percent lower risk of death compared to those with the lowest intake. Yet RCTs of supplementation with beta-carotene not only failed to confirm this benefit, they found an increased risk of cancer in the group with the highest intake. (2) Oops! Similar results have been found with vitamin E. (3)

Researchers Focus on Quantity, Not Quality

People don’t eat nutrition, they eat food. – Margaret Mead

The vast majority of observational studies today focus only on nutrients, isolated food components, or biomarkers—like saturated fats, carbohydrates, calories, LDL cholesterol—abstracted out of the context of foods, diets, and bodily processes.

This reductionist approach, which philosopher of science Gyorgy Scrinis calls “nutritionism,” has interfered with nutrition science’s ability to provide useful individual and public health guidance. (4)

The upside of nutritionism has been the discovery of drugs, vitamins, and minerals that have saved millions of lives. The downside is that Americans (and people all over the industrialized world) are obsessing over details like the percentage of fat or carbohydrates they consume rather than focusing on the broader and more important issues, like the quality of the food they eat.

Two examples of how this has manifested over the past few decades are:

• The promotion of margarine over the much better-tasting butter because of concerns about butter’s saturated fat content
• The vilification of eggs due to their cholesterol content without considering their overall nutrient value

(And of course, we now know that butter is healthier than margarine and dietary cholesterol has no impact on heart disease. Another oops!)

Nutritionism is a relatively new phenomenon. It started in 1977 with the McGovern Report, the first widely disseminated nutrition guidance to provide detailed, quantitative, nutrient-focused dietary recommendations. (5) Prior to that, dietary guidelines were based on familiar concepts of food groups and serving sizes and relatively simple information on what foods to buy and eat to maintain health. The average person could easily understand—and most importantly, act on—the guidelines.

After the McGovern Report, dietary guidelines became increasingly complex and difficult for the layperson to comprehend. The 1980 dietary guidelines were published in a short, 19-page brochure; in 1985 it grew to 28 pages; in 2010 it was 112 pages; and in 2015, the most recent dietary guidelines took up 517 pages!

What Happens When You Look at Food Quality

A more recent example of nutritionism can be found in the heated debate over whether low-fat or low-carb diets are superior for weight loss and metabolic and cardiovascular health. Each side of the debate has its advocates, and the controversy continues.

In early 2018, a group of researchers led by Dr. Christopher Gardner set out to settle this debate with an RCT. They assigned participants into two groups: low-carb and low-fat. But here’s the catch: they instructed both groups to:

1) maximize vegetable intake; 2) minimize intake of added sugars, refined flours, and trans fats; and 3) focus on whole foods that were minimally processed, nutrient dense, and prepared at home whenever possible. (6)

For example, foods like fruit juice, pastries, white rice, white bread, and soft drinks are low in fat but were not recommended to the low-fat group. Instead, the dietitians encouraged participants to eat whole foods like lean meat, brown rice, lentils, low-fat dairy products, legumes, and fruit. Meanwhile, the low-carb group was instructed to focus on foods rich in healthy fats, like olive oil, avocados, salmon, cheese, nut butters, and pasture-raised animal products.

Perhaps not surprisingly—if you don’t embrace nutritionism, that is—the researchers found that on average, people who cut back on added sugar, refined grains, and processed food lost weight over 12 months—regardless of whether the diet was low-carb or low-fat.

This was a fantastic example of what a nutrition study should look like. It resulted in clinically relevant, practical advice that is easy for people to follow: eat real food. Just imagine where we might be now if most nutrition studies over the past 40 years had been designed like this?

RCTs Are Better than Observational Studies but Still Problematic

If observational studies cannot prove causality, then why do they continue to form the foundation of dietary guidelines and public health recommendations? The answer is that RCTs also have several shortcomings that, thus far, have made them impractical as a tool for studying population health.

Duration

Most relationships between nutritional factors and disease can take years, if not decades, to develop. What’s more, the effects of some nutritional interventions in the short term are different than they are over the long term.

Weight loss is a great example. Both low-carb and low-fat diets have been shown to cause weight loss in the short term, but over the long term (more than 12 months) people tend to regain the weight they lost.

Inadequate Sample Size

The sample size, or number of participants in an RCT, is one of the most important factors in determining whether the results of the study are generalizable to the wider population. Most nutrition RCTs do not have a large enough sample size.

Dr. John Ioannidis, a professor at the Stanford School of Medicine, highlighted this problem in a recent editorial in BMJ called “Implausible Results in Human Nutrition Research.”

To identify a nutrition-related intervention that produces a legitimate 5 to 10 percent relative risk reduction in total mortality, we’d need studies that are 10 times as large as the highly publicized PREDIMED trial (which had around 7,500 participants), in addition to long-term follow-up, linkage to death registries, and careful efforts to maximize adherence.

RCTs Are Expensive

One reason that it’s such a huge challenge to design RCTs with sufficient duration and sample size is cost. RCTs are enormously expensive. In the pharmaceutical world, drug companies pay for RCTs because they have a vested financial interest in their results. But who will pay for long-term RCTs in the nutrition world? Public funding for nutrition research (and many other types of research) is declining, not increasing, which makes it unlikely that we’ll see long-term RCTs with sufficient sample sizes anytime soon.

Quality RCTs Are Difficult to Do

As Dr. Peter Attia points out in his excellent series Studying Studies, designing high-quality RCTs is fraught with challenges:

These trials need to establish falsifiable hypotheses and clear objectives, proper selection of endpoints, appropriate subject selection criteria (both inclusionary and exclusionary), clinically relevant and feasible intervention regimens, adequate randomization, stratification, and blinding, sufficient sample size and power, and anticipation of common practical problems that can be encountered over the course of an RCT.

That’s not an easy task and few nutrition RCTs meet the challenge.

Conflicts of Interest Are Very Common

It’s difficult to get a man to understand a thing if his salary is dependent upon him not understanding it. – Upton Sinclair

Many have written about financial conflicts of interest and their impact on all forms of research, including nutrition research. In short, research has shown that when studies are funded by industry, they are far more likely to report results that are favorable to the sponsor.

In one analysis performed by Marion Nestle, 90 percent of industry-sponsored studies returned sponsor-friendly results. (7) For a summary of the issues and how they impact the quality of nutrition research, I recommend this story from Vox.

In this article, I’d like to focus on another type of conflict of interest: allegiance bias, which is also known as “white hat bias.” Allegiance bias is not as well recognized as financial conflicts of interest are, which is one of the many reasons that it has an insidious effect on nutrition research.

Allegiance bias has been defined as “bias leading to distortion of research-based information in the service of what may be perceived as righteous ends.” (8)

For example, imagine that a vegan researcher sets out to do a study on the health impacts of a vegan diet. Is it possible that the researcher’s ideological commitment to veganism could influence, both consciously and unconsciously, how the study is designed, executed, and interpreted? Of course it could. In fact, it’s difficult to see how it couldn’t.

In a 2018 editorial called “Disclosures in Nutrition Research: Why It Is Different,” Dr. Ioannidis suggests that allegiance bias should be disclosed by researchers, just as financial conflicts of interest are. He says:

Therefore, it is important for nutrition researchers to disclose their advocacy or activist work as well as their dietary preferences if any are relevant to what is being presented and discussed in their articles. This is even more important for dietary preferences that are specific, circumscribed, and adhered to strongly. [emphasis added]

Ioannidis goes on to say that advocacy and activism, while laudable, are contrary to “a key aspect of the scientific method, which is to not take sides preemptively or based on belief or partisanship.” [emphasis added]

Veganism certainly meets the criteria of dietary recommendations that are “specific, circumscribed, and adhered to strongly.” In fact, some have pointed out that veganism meets the four dimensions of religion:

• Belief: Veganism began as a way to express moral integrity regarding the appropriation and suffering of non-humans.
• Ritual: Veganism involves strict dietary restrictions, including abstaining from the use of materials made from any animal products.
• Experience: The “holistic connectedness” of veganism would be considered a religious experience to those who live it.
• Community: There are many official and unofficial vegan associations across the world, and in 2017 a civil flag was created for the international vegan community.

Researchers and physicians like T. Colin Campbell, Kim Williams, Caldwell Esselstyn, Joel Fuhrman, John McDougall, and Neal Barnard could all be expected to suffer from this “white hat bias.” They’re involved in vegan advocacy and activism, both of which could be expected to be a source of allegiance bias.

A Famous Example of Allegiance Bias at Work

The China Study, a book by vegan physician and researcher T. Colin Campbell, is a perfect example. Campbell claimed that this study—which was not peer-reviewed—proved that:

• Animal protein causes cancer
• A plant-based diet protects against heart disease
• You can get all the nutrients you need from plants

Campbell even went as far as saying, “Eating foods that contain any cholesterol above 0 mg is unhealthy,” a claim that has been completely disproven and is reflected in the 2015 change in the U.S. Dietary Guidelines that no longer regards dietary cholesterol as a nutrient of concern.

However, since The China Study was published, several independent, peer-reviewed studies of the data have refuted T. Colin Campbell’s claims. For a great summary of the issues with The China Study, see this article by nutritional scientist Dr. Chris Masterjohn.

Allegiance bias can take several forms. It can involve:

• Cherry-picking studies to support a cherished view
• Misleadingly describing the results of studies that are cited in a paper
• “Data dredging” to search for statistical significance within given data sets (when no such significance is present)
• Not reporting null results
• Designing experiments for the purpose of obtaining a particular answer
• And more

It’s important to point out that allegiance bias is not always, or even often, conscious. Most researchers believe they are acting with scientific rigor and integrity. This is exactly why it’s so difficult to guard against, and why it’s so important to disclose.

Nutrition Policy Is Informed by Politics and Religion—Not Just Science

In a perfect world, dietary guidelines and nutritional policy would be the product of a thorough and dispassionate review of the available scientific evidence and not be unduly influenced by politics—and certainly not by religion. Dissenting views that are well informed would be not only welcomed but encouraged. As Syd Shapiro once said, “We should never forget that good science is skeptical science.”

Alas, we don’t live in a perfect world. In our world, dissenting views are are not welcomed; they’re suppressed. Dr. D. Mark Hegsted, a founding member of the Nutrition Department at the Harvard School of Public Health, made this opening remark in the 1977 McGovern hearing:

The diet of Americans has become increasingly rich—rich in meat, other sources of saturated fat and cholesterol … [and] the proportion of the total diet contributed by fatty and cholesterol-rich foods … has risen.

The only problem with this statement is that it directly contradicted USDA economic data which suggested that total calories and the availability of meat, dairy, and eggs at the time of the report were equivalent or marginally less than amount consumed in 1909. Full-fat dairy consumption was lower in 1977 than 1909, having declined steadily from 1950 to 1977. (9) Other evidence that contradicted Dr. Hegsted’s opinion was also ignored.

The feedback from the scientific community on the McGovern Report was “vigorous and constructive,” explicitly stated the “lack of consensus among nutrition scientists,” and presented evidence for the diversity of scientific opinion on the subject. (10) Other countries, such as Canada and Great Britain, also noted the lack of consensus on whether dietary cholesterol intake should be limited. U.S. senators issued the following statement about the McGovern Report:

It is clear that science has not progressed to the point where we can recommend to the general public that cholesterol intake be limited to a specified amount. The variances between different individuals are simply too great. A similar divergence of scientific opinion on the question of whether dietary change can help the heart illustrates that science cannot yet verify with any certainty that coronary heart disease will be prevented or delayed by the diet recommended in this report. (See footnote)

Nevertheless, these cautionary words were ignored, and the recommendations from the McGovern Report were adopted. This kicked off the fat and cholesterol phobia that would grip the United States for the next four decades.

Religion Can Impact Nutrition Guidelines

Another example of how non-scientific factors drive nutrition policy is the influence of the Seventh Day Adventists on public health recommendations in the United States and around the world. Seventh Day Adventists (SDA) is a Protestant denomination that grew out of the Millerite movement in the United States. Health has been a focus of SDA teachings since the inception of the church in the 1860s. According to Wikipedia:

Adventists are known for presenting a “health message” that advocates vegetarianism and expects adherence to the kosher laws, particularly the kosher foods described in Leviticus 11, meaning abstinence from pork, shellfish, and other animals proscribed as “unclean.” The church discourages its members from consuming alcoholic beverages, tobacco or illegal drugs. ... In addition, some Adventists avoid coffee, tea, cola, and other beverages containing caffeine.

Ellen White, an early SDA church leader, received her first major health reform vision in 1863, and “for the first time, God’s people were urged to abstain from flesh food in general and from swine’s flesh in particular.” Most SDA diet beliefs are based on White’s health visions.

White believed that the church had a duty to educate the public about health as a way to control desires and passions. Adventists continue to believe that eating meat stirs up “animal passions,” and that is one of the reasons for avoiding it.

Another early SDA leader, Lenna Cooper, was a dietitian who cofounded the American Dietetics Association, which continues to advocate a vegetarian diet to this day. Cooper wrote textbooks and other materials that were used in dietetic and nursing programs, not only in the United States but around the world, for more than 30 years. The SDA Church established hundreds of hospitals, colleges, and secondary schools and tens of thousands of churches around the world—all promoting a vegetarian diet—and played a major role in the development and mass production of plant-based foods, such as meat analogues, breakfast cereals, and soy milk. (11)

Adventists have been behind much of the early research on vegetarian diets at Loma Linda University in San Diego, where SDA leaders established a dietetics department in 1908. This was an ostensibly scientific endeavor at a university that was established by a religious group that believed vegetarianism was ordained by God.

If you think this raises a huge red flag for allegiance bias, you’re not wrong. In fact, as Jim Banta pointed out in a fascinating review of the SDA influence on diet, administrators at Loma Linda University in the mid-1900s initially discouraged research on vegetarian diets because “if you find the diets of vegetarians are deficient, it will embarrass us.” That is not the attitude of skepticism and open-minded inquiry that characterizes good science.

My Final Thoughts on Nutrition Research

I’d like to conclude with the opening two paragraphs of a recent open letter that scientists Edward Archer and Chip J. Lavie wrote to the National Academies of Sciences, Engineering, and Medicine:

“Nutrition” is now a degenerating research paradigm in which scientifically illiterate methods, meaningless data, and consensus-driven censorship dominate the empirical landscape. Since the 1950s, there was a naïve but politically expedient consensus that a person’s usual diet could be measured simply by asking what he or she remembered eating and drinking. Despite the credulous and unfalsifiable nature of this memory-based method, investigators used it to produce hundreds of thousands of publications and acquire billions of taxpayer dollars.

Over time, the sustained funding of demonstrably pseudo-scientific research methods has subverted the self-correcting nature of science and suppressed skeptical scholarship. Consequently, many decades of politics taking precedence over critical inquiry produced contradictory dietary guidelines, failed public policies, and the continued confusion over “what-to-eat.”

I couldn’t have said it better myself.

What do you think about the latest nutrition headlines? Do you read the newest research with skepticism? Let me know below in the comments—and be sure to check out Part 1 of this two-part series!

Staff of the Select Committee on Nutrition and Human Needs, United States Senate. Dietary Goals for the United States. 2nd ed. Washington, DC: Government Printing Office; December 1977.

The post Why You Should Be Skeptical of the Latest Nutrition Headlines: Part 2 appeared first on Chris Kresser.