by Judith Curry
When should research come with a ‘warning’ label?
Context
The issue of funding-induced bias in climate research has been addressed by these previous posts:
Specifically, the disclosure related to Willie Soon’s fossil fuel funding, and Representative Grijalva’s inquisition into skeptical climate researchers’ funding sources has raised a host of new issues for climate research to grapple with.
The fields of health/nutrition/food has long been wrestling with the issue of industry funding, some insights into the broader issues are raised in a serious of recent articles that are discussed below.
Conflicts of interest: Sugar and diet
PLOS Medicine published a recent article entitled Financial Conflicts of Interest and Reporting Bias Regarding the Association between Sugar-Sweetened Beverages and Weight Gain: A Systematic Review of Systematic Reviews. As the headline on Forbes.com put it, succinctly, “Big Sugar Tips The Balance Of The Research Scale.”
Another article, in the NYTimes: Coca Cola funds scientists who shift blame for obesity away from bad diets.
And for a different take: US govt actually encourages sugar consumption.
Harvard Business Review
The Harvard Business Review has a thoughtful article on the PLOS paper When research should come with a warning label. Excerpts:
Here is the problem with the PLoS study: it cannot answer the question it claims to answer because of the way it was designed. The researchers looked at systematic reviews conducted between 2006 and 2013. But this period saw a significant change in the kind of research done on sugar and weight gain. At the beginning of the period, there were few randomized control trials and a lot of observational studies; at the end, more randomized control trials, which provide much better evidence of cause and effect. The effect sizes for these trials are still quite small, but when added together, there was more, and qualitatively better, evidence associating sugar consumption and increased weight gain the closer one got to 2013. In fact, the four systematic reviews from 2012 onwards all found mostly positive associations; none were funded by industry.
One particular review is labeled as having a conflict with industry not because it was industry funded (it was funded by the National Institutes of Health) but because the authors had past industry funding from the food industry.
In fact, the people who would immediately know why the PLoS study is misleading are relatively few. Most would not have access to or even an interest in engaging the media, while many would be from industry, the very people the PLoS study effectively warns you not to trust.
Peer review is valuable, but it is not a guarantor of reliable knowledge. And while the editor of PLoS Medicine defends the paper in a preface by arguing that industry has particular reasons to be biased (“increased sales of their products”) but that academic researchers don’t (because they are engaged in “the honest pursuit of knowledge”), the distinction has become meaningless in the face of publish or perish academic pressures, publication bias toward positive findings, and the growing awareness that poor statistical methods and weak experimental design undermine so much academic research in medicine and health.
The key difference between academic and industry produced data is that we currently treat the second skeptically. That asymmetry is the flaw in our knowledge economy; it leads to moral hazard — and worse.
The PLoS findings may well be used to delegitimize legitimate scientific perspectives in future debates over sugared drinks and food in general. A cohort of scientists has been impugned, and a younger cohort warned that such is their fate if they work for industry. Like a malign butterfly flapping its wings furiously, the storm damage that a biased finding can do in a dynamic system of knowledge is considerable. And repairing it with the truth is very, very difficult.
Funding: Tales of Defamation
Nutrevolve has a provocative post entitled Funding: Tales of Defamation. Excerpts:
Wide generalizations are being made in the media lately, regarding nutrition and obesity researchers and where they get their funding. Often, these get to levels of insulting comparisons accusing researchers of being just like big tobacco scientists. The issue with the researchers is that they’ve taken money from Coke for their research, that their research pushes exercise over dietary intake as a cause of obesity and they pretty stupidly didn’t disclose the funding on their webpage until it was pointed out to them. While I don’t fully agree with their perspective on obesity, to claim that this is a front group and that these researchers are bought out is a far stretch. The lay perspective of this article appears to see little beyond the conspiracy, as evidenced by the comments. Even MedPageToday is telling readers that Coca Cola money has bought out these researchers.
I’ve been watching the industry funded research wars for a while now, and have popped in occasionally, particularly as the message has become less about improving research and talking about industry lobbying, and more about attacking independent researchers who have sought out industry funding. Often, I see little more than conjecture in accusations against researchers, and those shouting the loudest don’t help their case when they just point out funding and insinuate folly. It’s not surprising that some of the loudest voices in this field are those with no need for funding themselves – the few common names we see popping up and commenting have jobs/positions that rarely require the individual to be constantly writing grants in hopes of getting just one, and the reputation of a couple researchers’ is just a necessary casualty in the righteous crusade against industry funded research. Coke funding does not make the research bad, despite their corporate interests. The sacrifice of independent researchers’ reputations in this crusade is truly a shame, since Coke is going to run ads of thin people running with Coke bottles, regardless of whether or not scientists get funding from them.
We could talk forever and ever about what level of bias is instituted by industry funding, how we weigh that bias against the many other biases that occur in research, and whether that bias actually hurts the research in ways that things like peer review and editors can’t detect. We could also talk forever about whether we should be working with or against industry (or just flat out ignoring them). But these are likely conversations we’ll never get people to agree on.
What I want from those concerned about industry funding is to come off as genuine as possible. I challenge those concerned about industry funding to make a conscious effort to match every statement of concern about industry funding, with a call to action, to get taxpayers to write to their congressmen/women and advocate for higher NIH/NSF funding. Shout from the rooftops both your concerns, and the plight of researchers who write 14 grants in hopes of getting 1 or 2. At the very least, present the argument to the public that research costs money, and that if they don’t want it to be industry, they better be okay with it being their tax dollars.
This notion that researchers seeking industry money are doing conflicted research does little but subtly suggest that academic researchers find a new job or risk having their reputations threatened due to their funding source.
Do we only accept money from Quaker Oats and not the American Beverage Association? This can sound great on the surface, until you acknowledge that Quaker Oats is owned by Pepsi. The issue gets pretty complicated when you note that companies make many products e.g. do we not allow Coke funding because of their sugar sweetened beverages, even though they produce 100 percent fruit juices and diet beverages? Do we accept dairy money when they offer low fat milk but their product is also made into ice cream? Depending on who you ask, funding from the Whole Grain Council can be a sound choice or the biggest bias in the world. Do we only accept nutrient dense food producers, and soda producers are out because soda has no other nutrients besides sugars? Do we then not accept producers of foods like potatoes, that are ‘associated’ with more weight gain than are SSBs? I don’t particularly agree that there is righteous and non-righteous sources of funding.
It gets into a dangerously ambiguous realm to start saying that some types of industry funding are good but others are not. If anything, personal bias from researchers’ attachment to a theory seems the stronger hypothesis, as opposed to mindless drones/shills doing Coke’s bidding. The fact that Coke wants to toss a bunch of money at their hypothesis is just business as usual.
JC reflections
But . . . oil company funding. This is too often used as an excuse to reject a climate scientist or their findings, even if the funding is very indirect and has nothing to do with the specific study. For example, having accepted travel funds from a think tank that is in some way has some funding from an unacceptable industry group or individual can be game over for that individual.
In climate change research, there is no righteous source of funding – government funding can be a source of bias just as much as industry funding can, and there is A LOT more government funding out there. The need for greater intellectual (and political) diversity in climate change research has been addressed in this previous post.
That said, funding is probably a smaller source of bias than peer pressure to follow a consensus and to defend your own hypothesis, not to mention political preferences, environmental proclivities and career pressures.
In climate science, the ‘bogey’ is funding from fossil fuel companies. Well, regional power providers are also involved in wind power, solar power, geothermal and hydropower (not to mention nuclear, but not clear if nuclear is ‘good’ or ‘bad’?). Not to mention providing power for all those computers running weather and climate models. And where would the climate research elite be without fossil fuels to support their extensive air travel (its a badge of honor among them to be flying at least 100,000 miles per year). And is natural gas good, relatively good, or bad?
So . . . is funding from power and oil companies ok if it funds research related to wind, solar geothermal and hydro? Better predictions of extreme weather events that hamper both energy supply and demand, whatever the source of power? Or is it only a problem if it supports outreach efforts by a climate scientist to deny humans are the cause of climate change?
If independent scientists obtain funding from power and oil companies, would this help support needed intellectual diversity into climate science to avoid the massive groupthink we now see?
There is a lot we can learn by the extensive experiences and track record of the health/nutrition research interaction with industry funding.
At Georgia Tech, we are encouraged by the administration to interact with industry and get industry funding, particularly in this era of shrinking federal research dollars. Georgia Tech gets plenty of money from oil and power companies, although not nearly as much as heavy hitters such as Stanford, etc. [see Big Oil Goes To College] So . . . is funding from oil and power companies ok, as long as it isn’t used for climate research?
You can see that there is a lot of hypocrisy and stuff that simply doesn’t make sense. We need to have a serious discussion about bias in scientific research, and sources of funding is only one part of this discussion.
But witch hunts related to funding, even if unrelated to research, is a very disturbing trend, stay tuned for an interesting case from the GMO world.
Filed under: Ethics