What’s good for you seems to change every week. Maybe we should stop blaming the media and look at the studies underneath the stories, too.
Newspapers, magazines, or websites gives coverage of new studies on diet weekly. Fat might be good for you this week, and coffee is bad, then the following week’s article says fat is bad and coffee is good.
Researchers take a group of people, ask them questions about their diet and then relate their reported diet choices to outcomes like weight or cardiovascular health.
People don’t choose their diet randomly however.
In many cases, it is hard to see whether these problems will make food seem better or worse. Take coffee as an example. On the one hand, coffee drinkers are more educated on average, and they exercise more.
On the other hand, they also tend to smoke more. See the point? Depending on which factors these studies are able to adjust for, coffee can look good and bad.
Another example would be sugar substitutes. Standard sugar substitutes appear to encourage weight gain, while “fancier” plant-based substitutes cause weight loss.
Again, these patterns are hard to reconcile with the fact that neither of these have any calories but may be easier to understand when we see the links with education.
Organizing foods by who buys them is an extremely good way to understand the data overall.
The first problem is the selection in this setting is significant. The food choices aren’t random. They use food choices that are closely linked to other important characteristics of people that show differences in health.
It isn’t that controlling confounding variables don’t matter. They clearly do. The problem is that they’re not complete. There too many differences across people that we can’t adjust for. Even if we put more and more variables in the studies, there’s simply too much about people than to capture in data.
The quinoa effects are a good example for seeing why. Quinoa isn’t just an expensive food that highly educated people like. It’s also a food explicitly associated with health.
It’s eaten disproportionately by people like my parents, whose diet consists of edamame and Chinese long beans, constantly thinking about ways to be healthier. But there is simply no way that survey data is going to conclude that.
The same problem comes up in all kinds of health areas, and in other domains (child rearing, for example, or pregnancy, which I also write about).
On the media coverage side, I think there is more that can be done to be clear about the limitations of the studies we cover.
Randomized controlled trials will succeed in avoiding these problems by randomly assigning people to eat some particular foods. This gives accurate results. However, it’s time-consuming and expensive.
Researchers need to make more efforts to develop and use methods that allow us to think about the scope of the problems in this work. We need to be able to think more formally about the possible degree of bias that could result here.
14th February 2020 19:45