One of the longest standing Climate Audit issues with paleoclimate reconstructions is ex post decisions on inclusion/exclusion of data, of which ex post decisions on inclusion/exclusion of sites/data in “regional [treering] chronologies” is one important family. This was the issue in the original Yamal controversy, in response to which Briffa stated that they “would never select or manipulate data in order to arrive at some preconceived or unrepresentative result”. However, Briffa and associates have never set out ex ante criteria for site inclusion/exclusion, resulting in the methodology for Briffa regional reconstructions seeming more like Calvinball than science, as discussed in many CA posts.
Unlike Briffa, D’Arrigo has candidly admitted to the selection of data to arrive at a preconceived result. At the 2006 NAS panel workshop, Rosanne D’Arrigo famously told the surprised panelists that you had to pick cherries if you want to make cherry pie. Again in 2009 (though not noticed at the time), D’Arrigo et al 2009 stated that they could “partially circumvent” the divergence problem by only using data that went up:
The divergence problem can be partially circumvented by utilizing tree-ring data for dendroclimatic reconstructions from sites where divergence is either absent or minimal. (Wilson et al., 2007; Buntgen et al., in press; Youngblut and Luckman, in press).
Portfolio managers would have like to have a similar option in constructing portfolios: if, after the fact, you pick stocks that went up, it would be trivially easy to “circumvent” market downturns. That paleoclimatologists seem so obtuse to this simple observation is a major puzzlement.
In today’s post, I’ll show an absolutely breathtaking example of biased ex post picking by D’Arrigo et al in the D’Arrigo et al 2006 CNWT chronology. It was impossible for anyone to identify the full measure of this bias at the time or for many years afterwards, as D’Arrigo and coauthors failed to archive data at the time and refused to provide it when requested. They were supported in their refusal by IPCC WG1 Co-Chair Susan Solomon, who, as CA readers are aware, threatened me with expulsion as an IPCC AR4 reviewer for seeking supporting data for D’Arrigo et al 2006 (then cited in preprint by AR4). The data showing the cherry picking only became available in 2014 as part of a belated archiving program in the final year of Gordon Jacoby’s life.
I re-examined the Northwest Territories data about a year ago after discussing other Arctic series in eastern Canada and Greenland (Baffin Island, Gennaretti et al in Quebec) and turned to the next long data to the west. I’ve written up these notes because tree rings are back in the news with the publication of a new compilation of 54 series by Rob Wilson and coauthors that includes many Jacoby-D’Arrigo sites and a new article co-authored by Briffa (Schofield et al, 2016) which, while drawing attention to uncertainties in reconstructing temperature from chronologies, does no analysis of uncertainty/bias that can arise from ex post selection of data..
Coppermine River and Thelon River, Northwest Territories
Jacoby first sampled the Coppermine River site in 1977, later sampling Hornby Cabin (later re-named Thelon River), Mackenzie Mountains and Franklin Mountains in 1984. Three of these (Coppermine, Hornby Cabin and Mackenzie Mountains) were subsequently used in the Jacoby and D’Arrigo 1989 temperature reconstruction that was prominent in early NH reconstructions and as individual proxies in Mann et al 1998. Measurement data for the four sites was archived at ITRDB. Jacoby and D’Arrigo 1989 reported that they had considered data from 36 northern sites and selected the 10 sites “judged to provide the best record of temperature-influenced tree growth” (plus the very HS-shaped Gaspe site from south of the St Lawrence river). Jacoby refused to provide data from the sites that had been considered but not used (see CA post on his refusal here). The sample sizes at each site were relatively small: ~30-35 cores.
In 1999, D’Arrigo wrote a pleasant short recollection of the 1984 expedition to Thelon River. She closed the recollection wondering how the trees had responded to the warmer temperatures of the 1990s and 2000s and noting the value of updating the older chronologies – precisely the sort of update that seemed essential to me when I first encountered this field:
In 2004-5, D’Arrigo and others carried the contemplated update to the Thelon River (Hornby Cabin) and Coppermine River measurement data on a much expanded scale, this time obtaining over 350 cores from each location, as compared to ~30 cores in the earlier collection. Measurement data was not archived until 2014, but shows startling differences between the Thelon and Coppermine River sites: the Thelon chronology, in both RCS and STD versions, had a pronounced 20th century decline (divergence problem), whereas the Coppermine chronologies (especially the RCS version) had elevated late 20th century values. (The same phenomenon can also be observed in the chronologies from the 1980s.) The RCS series (red) have somewhat more low-frequency variability (but in RCS, such variability can arise from site inhomogeneity as well as temporal change.)
Figure 1. RCS (red) and STD (black) chronologies calculated (SM) from measurement (rwl) data archived in 2014 for Coppermine River (left) and Thelon River (right). STD chronologies were calculated from archived measurement data using the R-package dplR and the RCS chronologies were calculated using my emulation function, using a single standardization curve for each dataset.
D’Arrigo et al 2006: the CNWT Regional Chronology
D’Arrigo et al 2006 produced a NH temperature reconstruction from 19 regional chronologies. Their reconstruction was prominent in the NAS panel spaghetti graph in 2006 and in the 2007 IPCC AR4 and again in IPCC AR5.
The majority of the North American regional chronologies in D’Arrigo et al 2006 came from Jacoby and D’Arrigo’s own work, including the Central Northwest Territories (CNWT) regional chronology. In the figure below, I’ve excerpted the North American sector of the D06 location map (left panel), which shows the CNWT region as a large rectangle with four red location dots. Though not listed at the time, the two eastern CNWT locations are Coppermine River and Thelon River (both of which had large updates in 2004-5), while the two western sites were Franklin Mountains and Mackenzie Mountains, both of which only had limited “vintage” data available. On the right, I’ve shown a figure excerpt, showing the CNWT chronology as used in D’Arrigo et al (shown by them in SD units).
Figure 2. Left panel – excerpt from location map of D’Arrigo et al 2006; right panel – excerpt from D’Arrigo et al 2006 Figure ^, showing individual chronologies (in SD units).
Though the Coppermine and Thelon River measurement datasets were approximately equal in size, the 20th century portion of the D’Arrigo et al 2006 CNWT regional chronology does not appear to be impacted by the pronounced decline observed in Thelon River measurements.
This could not be asked as a question at the time, because, as noted above, D’Arrigo et al 2006 did not archive measurement data until many years later.
D’Arrigo et al 2009
D’Arrigo et al 2009 (Glob Plan Chg) was the first academic publication of the Coppermine River and Thelon River sites, though the sites had been used in chronologies and reconstructions for over 20 years. In this article, they made the statement about how to “partially circumvent” the divergence problem that I quoted above:
The divergence problem can be partially circumvented by utilizing tree-ring data for dendroclimatic reconstructions from sites where divergence is either absent or minimal. (Wilson et al., 2007; Buntgen et al., in press; Youngblut and Luckman, in press).
Although this was the only supporting academic publication for the CNWT series used in D’Arrigo et al 2006, they did not directly discuss this earlier chronology, other than in regard to the choice of RCS or STD standardization methods, about which they stated:
Previous analyses of Coppermine and Thelon ring-width data, in combination with ring width data from 2 other sites in the western NWT (R. Wilson unpublished report, 2005; D’Arrigo et al., 2006) revealed that the Regional Curve Standardization (RCS) method, a potentially useful technique for retaining centennial-scale climatic trends in tree rings (Briffa et al., 1992; Cook et al., 1995; Esper et al., 2002), did not appear to preserve any additional low-frequency information at these sites when compared to more traditional techniques. We thus did not utilize RCS for developing ring width chronologies for the present study, opting for more traditional methods (i. e. individual series detrending). We will, however, continue to explore efforts to capture more low-frequency variability in future studies using RCS and other methods.
While the decision between RCS and STD methodology is not central to this post, I note that the election between RCS and STD chronology in dendro articles is too often another ex post “researcher degree of freedom” in Wagenmakers’ usage. Curiously, in this case, the purported justification appears incorrect based on the appearance of the RCS and STD chronologies in Figure 1, in which the RCS chronologies appear to have more low-frequency variability than the STD chronologies (though this could arise from site inhomogeneity as well.)
D’Arrigo et al 2009 STD ring width chronologies for Coppermine River and Thelon River (see below). They have not archived these chronologies and did not archive the underlying measurement data until 2014. My own STD emulations using the data archived in 2014 correspond to these figures reasonably well.
Figure 3. Coppermine and Thelon ring width chronologies from D’Arrigo et al 2009 (Glob Plan Chg)
The Coppermine River chronology has a very similar 20th century uptick to the D06 CNWT regional chronology, while the Thelon River chronology has the familiar divergence problem.
Tracing the CNWT Cores
I was very puzzled by the much greater similarity of the CNWT “regional” chronology to the Coppermine River data than to the similarly sized Thelon River data.
One day in late 2014, many years after my initial request, I noticed that a considerable tranche of long overdue Jacoby/D’Arrigo measurement data had finally been archived, the long overdue housekeeping of Jacoby/D’Arrigo data being separately financed by the U.S. National Science Foundation. The data was archived in the last year of Jacoby’s life. (The archive regrettably continues to be incomplete: for example, MXD data from Labrador, later used in Wilson et al 2016, was withheld from the 2014 archive (and has not been archived in connection with the later publication.)
The measurement data as archived included a separate dataset for the 2006 D’Arrigo CNWT chronology (cana322.rwl) and for the 2003-2004 updates to Coppermine River (cana314.rwl) and Thelon River (cana318.rwl).
I carried out the somewhat laborious exercise of tracing the identification numbers of individual cores in the CNWT dataset (cana322.rwl) to component datasets, eventually locating all the cores. Some came from the Coppermine and Thelon River updates (cana314.rwl and cana318.rwl); some came from the vintage Jacoby datasets (Franklin Mountains, Mackenzie Mountains and Hornby Cabin.) Somewhat complicating the reconciliation, the vintage Coppermine River dataset (cana153.rwl) does not appear to have been incorporated into the updated dataset, but previously unarchived vintage data sampled in 1977 (with Jacoby site prefixes 813 and 825) were included in the update.
In Table 1 below, I’ve summarized the results of the reconciliation, showing the provenance of CNWT data from the Coppermine, Thelon and Western (Franklin Mts, Mackenzie Mts) locations, also showing available data not used. Rows distinguish vintage data from the recent updates. There was one astonishing result from the reconciliation (highlighted in yellow):
of the 359 (354+5) new Coppermine River cores, 98.6% were used in the CNWT composite and 1.4% not used, while of the 363 (12+351) new Thelon River cores, only 3.3% were used in the CNWT composite, while 96.7% were not used.
The reason why the CNWT looks so much like the (somewhat) upticking Coppermine series and so little like the (declining) Thelon is now easy to diagnose: D’Arrigo et al didn’t include the (declining) Thelon data in their composite
Notes: (1) Hornby Cabin (cana155) and Mackenzie Mts (cana156) used in Jacoby and D’Arrigo 1989 and Mann et al 1998; (2) Franklin Mts (cana154) not used in Jacoby and D’Arrigo 1989 or Mann et al 1998; (3) Cores with prefixes 813 and 825, sampled in 1977, not previously archived (to my knowledge); (4) Original Coppermine River archive (cana153), sampled in 1977, used in Jacoby and D’Arrigo 1989 and Mann et al 1998, but not included in CNWT composite.
D’Arrigo et al 2009 had stated that “the divergence problem can be partially circumvented by utilizing tree-ring data for dendroclimatic reconstructions from sites where divergence is either absent or minimal”. One could scarcely have expected that this would result in a situation where 98.6% of the cores of one site on the D’Arrigo et al 2006 location map were used, while 96.7% of the cores from another site shown on the location map were not used.
Impact on CNWT Chronology
The figure below shows the difference between the D’Arrigo et al 2006 STD chronology and an STD chronology calculated using all cores from datasets identified in the Table 1 reconciliation. The uptick in the D06 version no longer exists. (Bizarrely, the CNWT “RCS” chronology later archived by D’Arrigo et al was actually the STD chronology.)
Figure 4. Comparison of D’Arrigo et al 2006 chronology (red) to STD chronology (blue smooth of black) calculated using all cores in Table 1 datasets.
Conclusion
Seemingly arbitrary decisions by dendro specialists on inclusion/exclusion of tree ring data has long been a source of criticism at Climate Audit. In response to the original Yamal controversy about ex post decisions on inclusion of sites and withholding of adverse results, Briffa denied (later quoted by Guardian here) that they would select or manipulate data based on a “preconceived” result (a claim called in question by Climategate emails):
we would never select or manipulate data in order to arrive at some preconceived or unrepresentative result
Unfortunately, Briffa and associates have never set out ex ante criteria for site inclusion/exclusion, resulting in Briffa regional reconstructions seeming more like Calvinball than science, as discussed in many CA posts. However, remarkably, D’Arrigo et al 2009 (though not noticed at the time) had admitted earlier that year to doing exactly what Briffa had denied: the ex post selection of sites in order to obtain a preconceived result (a reconstruction that went up in the 20th century). They stated:
The divergence problem can be partially circumvented by utilizing tree-ring data for dendroclimatic reconstructions from sites where divergence is either absent or minimal. (Wilson et al., 2007; Buntgen et al., in press; Youngblut and Luckman, in press).
The full import of this sentence could obviously not be appreciated without knowing the full measure of the ex post cherry picking that had been carried out in the D’Arrigo et al 2006 CNWT reconstruction: who could possibly have guessed that D’Arrigo and coauthors used 98.6% of the new cores from the Coppermine River update (which went up) and only 3.3% of the new cores from the Thelon River update (which went down.) A more thorough implementation of a protocol for selection of data to achieve a preconceived result is hard to imagine.
In a separate post, I’ll examine NWT chronologies in subsequent multiproxy studies which did not use the D’Arrigo et al 2006 CNWT chronology but which nonetheless chose “advantageous” variations (PAGES2K, Wilson et al 2016).
Postscript on Jacoby and D’Arrigo 1989
As noted above, the Coppermine update included some vintage measurement data (prefixes 825 and 813) taken in 1977, but to my knowledge, not previously archived – though there is a vintage Coppermine River archive (cana153.rwl) with data also taken in 1977. No information is presently available on why the measurement data with prefixes 825 and 813 was not included in the original Coppermine River archive.
As an exercise, I calculated an STD chronology for the data with prefixes 813 and 825 (red below) and compared it to the archived cana153.crn chronology (black below), that had been used in the Jacoby and d’Arrigo NH reconstruction and later in Mann et al 1998. It is evident that there is a divergence problem in the previously unarchived data. One can not help wondering whether this was an earlier generation example of “circumventing” the divergence problem by “utilizing tree-ring data for dendroclimatic reconstructions from sites where divergence is either absent or minimal.”
Figure 5. Black – vintage Coppermine River chronology in ITRDB archive (1428-1977); red- STD chronology calculated from (previously unarchived) vintage data embedded in updated Coppermine River measurement data (314.rwl).
References:
D’Arrigo et al 2006. On the long‐term context for late twentieth century warming. JGR. pdf
D’Arrigo et al, 2009. Tree growth and inferred temperature variability at the North American Arctic treeline. Glob Plan Chg. pdf
Jacoby and d’Arrigo, 1989. Reconstructed Northern Hemisphere annual temperature since 1671 based on high-latitude tree-ring data from North America. Climatic Change.