Measuring bias in the U.S. federally-funded climate research

by David Wojick
Semantic analysis of U.S. Federal budget documents indicates that the climate science research budget is heavily biased in favor of the paradigm of human-induced climate change.

For decades climate research has been dominated by a paradigm that posits dangerous, human-induced global warming. This concept is usually referred to as “anthropogenic global warming” or simply AGW. The competing paradigm, which posits the possible attribution of significant natural variability, is barely mentioned. We call this bias “paradigm protection.”
We developed a method to quantify this paradigm protection bias, a method with general applicability in bias research. See our Framework Working Paper for a detailed discussion of bias quantification issues.
We first define two sets of words that express core concepts for each paradigm :

  1. human induced climate change; and
  2. the climate change attribution problem.

Then we measure the rates of occurrence of these two sets of words in the budget documents. The occurrence ratio we find is about 80 to one in favor of the human-induced paradigm. Moreover, it is roughly constant across multiple budget reports, a clear indication of paradigm bias.
Thomas Kuhn, in his groundbreaking work on the structure of scientific revolutions, coined the word “paradigm” to describe the basic tenants that guide research in a given domain. He pointed out that a scientific community may actively ignore new ideas that challenge its paradigm. We have coined the term “paradigm protection” to describe this behavior.
Given the situation in climate science, with new ideas challenging the dominate AGW paradigm, it is reasonable to think that some paradigm protection may be occurring. We have already observed that the National Science Foundation is ignoring the attribution problem (see NSF in Climate Denial?).
Our semantic analysis is designed to test for paradigm protection in the $2.5 billion per year U.S. Global Change Research Program (USGCRP). The basic approach is to look for language that expresses the dominant paradigm, versus language indicating that the challenging ideas are under discussion. In this case we simply chose fifteen words that reflect each side in the scientific paradigm debate. The occurrence of these words is a tentative measure of the extent to which the corresponding ideas are under consideration.
The underlying principle is that specific ideas require specific language for their expression. We then measure the occurrence of these words in three successive USGCRP budget reports, spanning five fiscal years.
The AGW paradigm
Our search terms are chosen to reflect key features of the AGW paradigm. There is an emphasis on modeling the supposed adverse impact of human emissions of CO2 and other greenhouse gases. The physical focus is on radiation physics, plus the role of clouds and aerosols. The resulting risks include droughts, floods and disease. We refer to these words as AGW centric.
Our 15 AGW centric search terms are these: Model, Carbon, Impact, Risk, Cloud, Forecast, Predict, Aerosol, Flood, Carbon Dioxide, Greenhouse gas, Radiation, Sea level, Disease and Heat Wave.
The attribution problem challenger
In the last decade or so the AGW paradigm has increasingly been challenged by an alternative paradigm, which emphasizes what is called the attribution problem. The question is to what degree natural climate variability is a significant cause of global warming over the last century or so? How much of the warming should be attributed to natural causes versus human causes? It is argued that this problem is unsolved, so it should be central to the research.
Our search terms therefore focus on features like ocean oscillation and solar variability, which are two of the proposed primary natural drivers. There are also issues like emerging from the little ice age and the existence of prior natural warm periods, as well as the role of chaotic oscillation. We refer to these words as attribution problem centric.
Our attribution problem centric search terms are these: Solar, Natural variability, Natural flux, Ocean circulation, Pacific Decadal Oscillation, North Atlantic Oscillation, Solar variability, Solar cycle, Sunspot, Little Ice Age, Warm period, Natural Warming, Chaotic, Chaos and Cosmic rays.
Analysis of USGCRP budget documents
The USGCRP includes almost all of the climate related basic scientific research being performed by US Federal agencies, including NSF. The budget is about $2.5 billion a year, which is a significant fraction of worldwide climate research.
The question is to what extent is the USGCRP exploring the attribution of global warming to recent long-term natural variability, as opposed to AGW? Our approach has been to take a series of their annual budget reports — “Our Changing Planet” — and search each using the two sets of stem words that reflect the two competing paradigms. Our Changing Planet provides extensive discussion of the research program, so it is reasonable to believe that it is representative of that program.
We searched the FY 2012, FY 2014 and FY 2016 editions, for each of our stem words. In each case we simply logged the total number of occurrences of all of the words that included the stem word. So for example search on the stem word “model” includes all occurrences of the words “modeled,” “modeling” and “modeler” as well as “model.”
What we have found
The statistical results of this analysis are dramatic. For the total of all 15 AGW centric stem words we found a range of about 7 to 9 occurrences per page on average. Specifically we found a total of 644 occurrences in 78 pages (8.3 per page) for FY 2016, then 340 occurrences in 49 pages (6.9 per page) for FY 2014 and 482 occurrences in 53 pages (9.1 per page) for FY 2012.
In contrast we found just 0.1 occurrences per page for the attribution problem centric stem words. For the total of all 15 attribution problem centric words we found just 7 occurrences in 78 pages (0.1 per page) for FY 2016, just 5 occurrences in 49 pages (0.1 per page) for FY 2014, and just 4 occurrences in 53 pages (0.1 per page) for FY 2012.
In short there appears to be virtually no discussion of the natural variability attribution idea. In contrast there appears to be extensive coverage of AGW issues. The ratio of occurrences is roughly 80 to one. This extreme lack of balance between considerations of the two competing paradigms certainly suggests that paradigm protection is occurring. Apparently it is not just NSF that is ignoring the attribution problem; rather it is the entire USGCRP.
AGW-centric modeling
The detailed AGW word counts also provide valuable information regarding the specific focus of the USGCRP. That there is almost no consideration of attribution to natural variability is quite clear. The highest count terms for FY 2016, 2014 and 2012, respectively, are these:

  • Model 112, 71, 185
  • Carbon 106, 19, 30
  • Impact 101, 69, 72
  • Predict 74, 36, 58

The strong, central focus of the USGCRP is to use models to try to predict the impact of changes in atmospheric carbon. All other considerations are secondary. It is no accident that variants of the word “model” dominate these documents. Understanding global warming and climate change are not the focus of Federal climate research. It is all about using models to make scary predictions.
Of course solving the attribution problem will involve modeling as well, so the term “model” is not AGW centric in principle. But it is clear from the extremely low attribution word counts that attribution modeling of natural variability is not under serious consideration. The modeling actually being done is heavily AGW centric. Moreover, this modeling dominates climate research. See our semantic analysis of this here: http://www.cato.org/blog/climate-modeling-dominates-climate-science.
Conclusion
This bias in favor of AGW has significant implications for US climate change policy. Present policy is based on the AGW paradigm, but if a significant fraction of global warming is natural then this policy may be wrong. Federal climate research should be trying to solve the attribution problem, not protecting the AGW paradigmFiled under: Sociology of science

Source