by Judith Curry
Siddhārtha Gautama was a prince who was only told good news, and protected from seeing suffering and death. But he finally realised that he was not seeing the world as it really was, and so he left his palace to first take on the life as a wandering ascetic, and eventually to become the Buddha. – David Spiegelhalter
Dominic Lawson published an article in the Times (2014) Listen to Dr Drivel and you are sure to develop a condition. Unfortunately it is behind paywall, but this article is discussed in an essay by David Speigelhalter: A heuristic for sorting science stories in the news. Excerpts:
Dominic Lawson’s article in the Sunday Times today quotes me as having the rather cynical heuristic: “the very fact that a piece of health research appears in the papers indicates that it is nonsense.” I stand by this, but after a bit more consideration I would like to suggest a slightly more refined version for dealing with science stories in the news, particularly medical ones.
The immediate impulse behind Lawson’s article was a spate of studies claiming associations between ordinary daily habits and future bad outcomes: eating a lot of white bread with becoming obese, being cynical with getting dementia, light bedrooms with obesity (again). All these stories associate mundane exposures with later developing dread outcomes, i.e. the classic ‘cats cause cancer’ type. My argument is that, since we would not be reading about a study in which these associations had not been found, we should take no notice of these claims.
Why my cynicism? There has been a lot of public discussion of potential biases in the published scientific literature – see for example, commentaries in the Economist and Forbes magazine. The general idea is that by the time research has been selected to be submitted, and then selected for publication, there is a good chance the results are false positives: for a good review of the evidence for this see ‘A summary of the evidence that most published research is false’. There is also an excellent blog by Dorothy Bishop on why so much research goes unpublished.
The point of this blog is to argue that such selection bias is as nothing compared to the hurdles overcome by stories that are not only published, but publicised. For a study to be publicised, it must have
• Been considered worthwhile to write up and submit to a journal or other outlet
• Have been accepted for publication by the referees and editors
• Been considered ‘newsworthy’ enough to deserve a press release
• Been sexy enough to attract a journalist’s interest
• Got past an editor of a newspaper or newsroom.
Anything that gets through all these hurdles stands a huge chance of being a freak finding. In fact, if the coverage is on the radio, I recommend sticking your fingers in your ears and loudly saying ‘la-la-la’ to yourself.
The crucial idea is that since there is an unknown amount of evidence that I am not hearing about and that would contradict this story, there is no point in paying attention to whatever it is claiming.
I have been struggling to find a suitable name for this heuristic, perhaps with some literary or classical allusion to someone who was misled by only being told selected items of information. Perhaps the ‘Siddhartha’ heuristic? Siddhārtha Gautama was a prince who was only told good news, and protected from seeing suffering and death. But he finally realised that he was not seeing the world as it really was, and so he left his palace to first take on the life as a wandering ascetic, and eventually to become the Buddha.
JC comments: I just spotted this article that I put in my file over a year ago. There are several aspects of relevance to the alarmism associated with climate change.
Spiegelhalter’s point about science that makes it into the media is very consistent with a point I made last week in the post Which climate papers ‘matter’?:
More than 25% of the top 100 papers were published by Nature Climate Change. Nature Climate Change is clearly going for the headlines/altmetrics, with the unfortunate result that a substantial fraction of their highest profile papers don’t even survive their press release.
Analogously to the alarming medical headlines warning future bad outcomes associated with ordinary daily habits, we see alarming news articles warning of future catastrophes related to climate change.
Spiegelhalter raises an important point: the manufactured and enforced consensus on climate change results in an unknown amount of evidence that we are not hearing about that would challenge the consensus.
And finally with regards to the Siddhartha heuristic: Siddhartha was misled by only being told selected items of information, but the selected information was distinctly anti-alarmist and he heard only the good news. With health research and climate change science, we are hearing a preponderance of bad news. The motivations for the alarming health research is presumably tied to building reputation and funding base, amplified by the media’s desire for an attention-grabbing story. The climate change alarm is further motivated by building political will to act on reducing emissions.
The key challenge, as the Paris confab approaches, is to see the world – its climate and its myriad of problems and challenges – as it really is. And not to view it through a narrow lens that focuses only on climate alarmism and demonizing fossil fuels.
Filed under: Communication, Sociology of science