by Judith Curry
Who assesses the assessors of climate science research? A new paper reviews the climate change reviewers by comparing references in the NIPCC and IPCC reports.
In September 2011, I had a post NIPCC discussion thread. The NIPCC provides a report that parallels the IPCC assessment report in structure, but comes to starkly different conclusions.
A very interesting new paper has been published that explores some of the differences between the IPCC and NIPCC reports, and asks the question:
Should we take the ‘contrarians’ and their arguments seriously or not?
Reviewing the climate change reviewers: Exploring controversy through report references and citations
Ferenc Jankó, Norbert Móricz, Judit Papp Vancsó
Abstract. There is a growing need to analyse the knowledge controversies about climate change. Human geography has a role in understanding of the motivations and sources of the participants in the debate. In this study, we explore the scientific background of the contrarian arguments, using Climate Change Reconsidered published by the conservative think tank Heartland Institute, in comparison with the Fourth Assessment Report of the Intergovernmental Panel on Climate Change The Physical Science Basis. Firstly, we surveyed the reference lists, which showed that in general the contrarian report used the same journals, as their most important sources. However, the differences are in the details: journals dealing with paleo-issues are more important for the contrarian report. Further, it is noteworthy that we found only 262 identical references (4.4% of all references) in the reports and their contextual analyses revealed that the rhetoric can be remarkably different, as can the way in which an article is used. These results indicate that we cannot state that the opponents use completely different sources, but the complementarity of their reference list raised some questions which are discussed in the last section of the paper. Should we take the ‘contrarians’ and their arguments seriously or not?
Published by Geoforum [link] to abstract.
Excerpts:
Philosophy and sociology of science offer some simple starting points to explore and understand the debate over climate change. The ‘scientific field’ concept of Bourdieu presents the controversy as mainstream science protecting the field of climate change from the attacks of the contrarians. According to the concept it may be suggested that the ‘field’ of climate change is not homogeneous; it has a changing structure, its agents have different amounts of scientific capital, and the boundaries of the field are continually being re-demarcated.
Studying climate change controversies therefore was not in the research focus of either science studies, or geography. Considering the debates in principle, it is difficult to bring climate change and science studies together. Paradoxically, science studies are traditionally identified with the ‘Academic Left’, while climate sceptics are usually connected to conservative ideology and politics. ‘‘Though there are no substantive connections’’ between them, the arguments of climate sceptics ‘‘exemplify many of the wider claims made in academic science studies about the construction of scientific knowledge’’. This situation is quite inconvenient for scholars of science studies. ‘‘Was I wrong to participate in the invention of this field known as science studies? Is it enough to say that we did not really mean what we said? Why does it burn my tongue to say that global warming is a fact whether you like it or not? Why can’t I simply say that the argument is closed for good?’’ pondered Latour (2004).
The dispute around anthropogenic climate change is also an emergent field for geography and related thought (Hulme, 2009) as well as the history of climate change science.
Setting the references and in-text citations at the focus of our study, our research questions were the following:
- Analysing the reference lists, what difference is there in the scientific basis of CCR compared to the IPCC?
- Considering the identical references, what is the difference in how they are used (e.g. interpretation and context)?
- How does the rhetoric of the sceptical report differ from that of the IPCC report?
From the Conclusions:
‘‘The debate has only just begun’’ concluded Grundmann in his recent paper about the legacy of Climategate. Indeed, our results raise some further questions about the knowledge controversy escalating after the email incident. Analysing the difference between the reference lists, we concluded that scientific arguments were constructed from the similar material; references came mainly from the same journals and the same journals were among those most cited by both sides. Should we take the contrarian statements seriously? Should we consider them as well-established statements, legitimated with the same or similar peer reviewed journals? If we say ‘no’ and take the CCR (NIPCC) as a partisan report, produced by cherry-picking the literature (a mutual charge in the controversy), there is no end to the debate.
Our results show that some of the difference lies in the details; journals dealing with paleo-issues are more important for the NIPCC report. This raises a cautious question: is there an opposition in climate science palpable between paleoclimatology and the mainstream methods, similarly to the difference between observation and modelling? In other words, are there specific sub-fields in climate science, which provide some more evidence against the anthropogenic climate change idea? Another question is who are the authors of the references used by the contrarian report? We checked the original references only in some cases but we did not analyse these scholars. Only further research can give the answer to both questions.
Based on the above findings and because grey literature had only a small significance in both reports, we cannot state that climate sceptics use completely different sources to demolish the architecture of mainstream climate science. Thus, we should reject the assumption that the reference list of CCR (NIPCC) would differ markedly from that of the IPCC report. On the contrary, the contextual and rhetorical analysis of the extreme weather and paleoclimate chapters and sub-chapters revealed that not only do the contrarians have their key-authors, but so does ‘mainstream’ science and it was very instructive to see the pre-organised battle of these references with the pre-assigned winners, who were supporting the knowledge claims of the reviewers and ‘weakening the enemies’.
Further, not only the facts, but the readings of the same facts differ, which makes the assessment process flexible. What are the implications for science? There is a real concern that the controversy has so far had a negative effect on the reputation of science. From the perspective of an idealised public view of science, such a polarised debate about ‘truths’ may be confusing. Thus, social science with science studies in the forefront has a mission to change this obsolete view of science. Saying ‘yes’ to our first question we might have a somewhat ‘naive’ implication for the IPCC; improving and widening the reviewing process may be a possible answer to the contrarian criticisms. But when we take the contrarian arguments seriously, there is a chance to bring together the differing views and knowledge claims of the disputing ‘interpretive communities’.
Scientific reports should be viewed not only as a second level of peer review and canonization of scientific facts but also as a means of politicization of science. [T]here will be hope for better science for the public and for policy, for better constructions of the problem only when we fully understand the knowledge controversy around climate change.
JC reflections
First, it is very good to see the sociology of climate science moving away from the the reflexive defense of the consensensus position (e.g. the social psychology of deniers), to take a serious look at the climate controversy. By comparing the IPCC and NIPCC reports, the authors take on a very interesting topic but have only scratched the surface.
The paper reminds us that such assessment reports are a means of politicization of science. While on the surface the NIPCC might seem more open to this charge, the shoe fits the IPCC as well.
The statement: Not only the facts, but the readings of the same facts differ, which makes the assessment process flexible. This reminds of the challenges of reasoning about climate science complexity and uncertainty [link to Reasoning about climate uncertainty]
The concluding point is important: But when we take the contrarian arguments seriously, there is a chance to bring together the differing views and knowledge claims of the disputing ‘interpretive communities’.
I look forward to future papers on this topic.
Filed under: Skeptics, Sociology of science