by Judith Curry
If you think about the costs I’ll pay for raising these concerns, including the cost of damaged relationships with people that I like, I think you will conclude that a personal commitment to science is the only thing that could be big enough to offset these costs. – Paul Romer
Ed Dolan pointed me to a series of remarkable posts from economist Paul Romer’s blog:
- Freshwater feedback Part I: Everybody does it.
- Feynman integrity
- Stigler conviction versus Feynman integrity
Romer starts with a story about freshwater economists, the details of which are not particularly germane to what I find interesting in this. My excerpts focus on issues raised by Romer that are of broader relevance to the climate debate. Excerpts:
Together, the evidence I summarize in these three posts suggests that freshwater economists differ sharply from other economists. This evidence strengthens my belief that the fundamental divide here is between the norms of political discourse and the norms of scientific discourse. Lawyers and politicians both engage in a version of the adversarial method, but they differ in another crucial way. In the suggestive terminology introduced by Jon Haidt in his book The Righteous Mind, lawyers are selfish, but politicians are groupish. What is distinctive about the freshwater economists is that their groupishness depends on a narrow definition of group that sharply separates them from all other economists. One unfortunate result of this narrow groupishness may be that the freshwater economists do not know the facts about how most economists actually behave.
JC comment: ‘groupishness’ seems like the ‘tribalism’ among some climate scientists that I wrote about in the aftermath of Climategate, and how this led to behavior that put them outside the norms of science.
In my informal conversations, no one from the freshwater camp has articulated exactly what they mean by the adversarial method, so I’ll try to fill in the blanks here. In an equilibrium where everyone follows the adversarial method, each side tries to make the best possible case for its position. What we might call the rules of evidence are that an advocate cannot make statements that are false, but it is to be expected that an advocate will withhold information that does not support the advocate’s position.
The freshwater sympathizers agreed, for example, that Lucas and Moll strategically refrained from verbal disclosures about some of the properties of the underlying mathematical formalism. Where we disagreed was whether this was a sign of behavior by the authors that is wrong. In effect, their response was caveat emptor; this is what all economists do.
One way to characterize the underlying disagreement about what is wrong is to say that we are both commenting on strategic interaction between economists that takes the form of a repeated, multi-player prisoner’s dilemma. In this game, following the scientific method corresponds to cooperation; following the adversarial method corresponds to defection. My claim is that economics is characterized by a trigger strategy/reputational equilibrium that sustains the scientific method. In calling attention to defection by the freshwater economists, I am following a strategy that is equilibrium play in this reputational equilibrium.
In contrast, the freshwater economists believe that we are already in the noncooperative adversarial equilibrium, so it is wrong to express disapproval of economists who are simply engaging in the type of behavior that is rational in that equilibrium. The freshwater economists might agree that in some first-best sense, an equilibrium based on the scientific method would be preferable, but they apparently believe that we are not in such an equilibrium; that it is not possible to get back to such an equilibrium; and that even if we did, it would not be possible to sustain it.
If so, what looks superficially like a deep and intractable disagreement about values or morality may simply reflect disagreement over the facts about what most economists do. When the freshwater types say “everybody is following the adversarial method,” what they may honestly be saying is that “everybody I know is following the adversarial method and they all believe that everyone else is doing this too.” But because freshwater economists have so sharply isolated themselves from the rest of the profession, they may be generalizing from an unrepresentative set of observations.
JC comment: I find Romer’s articulation of ‘adversarial science’ to be quite insightful and clearly describes what is going on in certain sectors of climate science. It seems that the motivations for adversarial science may be to support policy advocacy, prop up professional egos, and/or careerism.
Feynman integrity
In a previous post, I said that my claim about mathiness could be reduced to two assertions:
- Economist N did X.
- X is wrong because it undermines the scientific method.
I reported that in conversations with economists I referred to loosely as freshwater sympathizers, I found agreement on 1 but disagreement on 2. Specifically, I heard two things:
- a) Yes, but everybody does X; that is how the adversarial method works.
- b) By selectively expressing disapproval of this behavior by the freshwater economists that you name, you, Paul, are doing something wrong because you are helping “those guys.”
JC comment: ‘helping those guys’ is an unpardonable ‘sin’, as Lennart Bengtsson and others have discovered.
If you know me and think about the costs and benefits I’m facing, I think you will conclude that the most likely explanation for my decision to raise these issues is that I am genuinely trying to come to grips with what has gone wrong in macro-economics and that I am truly committed to science as the noblest human achievement. If you think about the costs I’ll pay for raising these concerns, including the cost of damaged relationships with people that I like, I think you will conclude that a personal commitment to science is the only thing that could be big enough to offset these costs.
JC comment: WOW. What a statement. I wish I had said this.
From a commencement address by Richard Feynman that described what this type of integrity entails:
It’s a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty–a kind of leaning over backwards. For example, if you’re doing an experiment, you should report everything that you think might make it invalid–not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you’ve eliminated by some other experiment, and how they worked–to make sure the other fellow can tell they have been eliminated.
Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you can–if you know anything at all wrong, or possibly wrong–to explain it. If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it. There is also a more subtle problem. When you have put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition.
Call this Feynman integrity.
JC Comment: Always delighted to have an excuse to post a Feynman quote (and this one is his best, IMO). In my Uncertainty Monster paper, I included parts of this particular quote.
Stigler conviction
In a previous post, I referred to Feynman Integrity as one possible guide to a life in science. An alternative is what I’ll call Stigler Conviction (Stigler 1955, “The Nature and Role of Originality in Scientific Progress,” [link]).
The techniques of persuasion also in the realm of ideas are generally repetition, inflated claims, and disproportionate emphases, and they have preceded and accompanied the adoption on a large scale of almost every new idea in economic theory. Almost, but not quite, every new idea. A few men have such unusual powers that their contemporaries recognize their claims without the usual exaggerations: Smith and Marshall are the only economists who seem to me indisputably to belong in this supreme class.
The rest have employed in varying degrees the techniques of the huckster. Although the new economic theories are introduced by the technique of the huckster, I should add that they are not the work of mere hucksters. Indeed I do not believe that any important economist has ever deliberately contrived ideas in which he did not believe in order to achieve prominence: men of the requisite intellectual power and morality can get bigger prizes elsewhere. Instead, the successful inventor is a one-sided man. He is utterly persuaded of the significance and correctness of his ideas and he subordinates all other truths because they seem to him less important than the general acceptance of his truth. He is more a warrior against ignorance than a scholar among ideas.
Nor do I argue that a strong conviction of the validity of one’s ideas and energetic dissemination are sufficient to alter significantly a science’s work. It is possible by mere skill of presentation to create a fad, but a deep and lasting impression on the science will be achieved only if the idea meets the more durable standards of the science. Among these standards is truth, but of course it is not the only one.
JC comment: Hucksterism is a great word to describe what goes on in the communication of climate science in service of policy advocacy. The complicity of many climate scientists and professional societies in this hucksterism is a cause of great concern.
The path that led Lucas and his followers to increasingly implausible positions defended using increasingly adversarial arguments starts with Stigler conviction and a commitment to an initial conjecture that turned out to be false. Why did Lucas, who as far as I can tell was originally guided by Feynman integrity, switch to the mode of Stigler conviction? My conjecture is economists let small accidents of intellectual history matter too much. If we had behaved like scientists, things could have turned out very differently.
JC comment: The year 2000, the publication of the IPCC TAR, and the elevation by IPCC Chair John Houghton of the hockey stick to icon status. A small accident in the history of the IPCC has had serious adverse consequences for climate science.
It is worth paying attention to these accidents because doing so might let us take more control over the process of scientific inquiry that we are engaged in. At the very least, we should try to reduce the odds that that personal frictions and simple misunderstandings could once again cause us to veer off on some damaging trajectory. I suspect that it was personal friction and a misunderstanding that encouraged a turn toward isolation (or if you prefer, epistemic closure) by Lucas and colleagues. They circled the wagons because they thought that this was the only way to keep the rational expectations revolution alive.
JC comment: The tragic case in point for climate science is Mann versus McIntyre, as revealed by Andrew Montford and the Climategate emails. ‘Circling the wagons’, even. I’ve written previously of how we managed to quickly get back on track on the hurricane and global warming wars, whereas Mann continues to fight the hockey wars not just by hucksterism but by attacking his opponents. This kind of behavior does not help keep the dangerous human caused climate change narrative alive, and at some point simply becomes pathological.
The misunderstanding is that Lucas and his colleagues interpreted the hostile reaction they received from such economists as Robert Solow to mean that they were facing implacable, unreasoning resistance from such departments as MIT. In fact, in a remarkably short period of time, rational expectations completely conquered the PhD program at MIT.
JC comment: Climategate and Gleick-gate are cases in point here (and far more pointless than Romer’s example), with perceived exaggerated threats from the likes of Heartland and Warwick Hughes. Why trash their integrity when the IPCC had received the Nobel Peace Prize and they were claiming 97% consensus?
JC reflections
I find Romer’s posts to be very insightful, and to be an important contribution to the discussion on advocacy by scientists (for previous CE posts, see [link]. The issue is NOT that scientists have values, or even express them. Rather the problem is engaging in adversarial science in support of these values, whereby their public communications focus on repetition, inflated claims, and disproportionate emphases.
How we can bring Feynman integrity back to policy relevant climate science is a considerable challenge. Climategate was a watershed moment in that it turned the tide slightly in the direction of discussing uncertainty in the public debate on climate change. Given the extremely high policy relevance of climate science, this transition to Feynman integrity will require a better decision analytic model than the linear model that ‘speaks consensus to power’ – examples of such strategies are provided in these previous CE posts.
p.s. I’ve tried to keep the Mann references to a minimum, although it wasn’t easy given that I have just finished reading Mark Steyn’s new book A disgrace to his profession: The world’s scientists on Michael E. Mann, his hockey stick and their damage to science. Stay tuned for a post on this. In the long run, I don’t think adversarial science and hucksterism will ‘pay’.
Filed under: Ethics, Scientific method