John Cook, proprietor of the grossly mis-named website, SkepticalScience, published a study in which Cook claims, “A new survey of over 12,000 peer-reviewed climate science papers by our citizen science team at Skeptical Science has found a 97% consensus among papers taking a position on the cause of global warming in the peer-reviewed literature that humans are responsible.” That’s the headline picked up by the gullible press.
Unlike what would be implied by the name, SkepticalScience is a proponent of catastrophic anthropogenic global warming (AGW).
The study involved reading the abstracts from 11,944 papers which contained the words “global warming” or “global climate change” published in the period 1991-2011. The “team” rated these papers according to seven criteria judging their support for AGW, see definitions of the categories in table 2 of Cook’s published paper here.
The actual findings of the survey, in Cook’s own words: “We find that 66.4% of abstracts expressed no position on AGW, 32.6% endorsed AGW, 0.7% rejected AGW and 0.3% were uncertain about the cause of global warming.” So how do you get a 97% consensus out of that?
That question has been discussed on many blogs. The best explanation I’ve found is one by Dutch researcher Marcel Crok, “Cook’s survey not only meaningless but also misleading.“
Here are some excerpts [I made minor edits of spelling]:
Cook et al. selected around 12,000 scientific abstracts that contained the words “global warming” or “global climate change” published in the period 1991-2011. With a large group of volunteers they then rated the papers using 7 categories. Around 8000 of the abstracts (2/3) take no clear position on Anthropogenic Global Warming (AGW). Of the remaining ~4000 abstracts more than 97% “endorse AGW” according to the paper. Only a tiny amount (78 papers) “reject AGW”. Hence they claim again that there is a consensus, that the debate is over and also that there is a gap between scientists and the public.
Now here comes the misleading part. If an abstract/paper “endorses AGW”…[this means that] change is real, man-made and dangerous.
To the credit of the researchers they made all their results available in a searchable database. Their rating system is online as well. There are 7 levels of endorsement, going from quantified endorsement of AGW all the way down to a quantified rejection of AGW. Seems fair enough. But here is the issue. Only the first category can be regarded as a real or strong endorsement of AGW. Here is the description of category 1:
1. Explicit Endorsement of AGW with quantification
1.1 Mention that human activity is a dominant influence or has caused most of recent climate change (>50%).
1.2 Endorsing the IPCC without explicitly quantifying doesn’t count as explicit endorsement – that would be implicit.
Now specifically look at 1.1. This comes close to the iconic statement from the IPCC AR4 report which said that “Most of the observed increase in global average temperatures since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations.” Now if 97% of the abstracts would repeat in slightly varying terms this major conclusion, then at least the conclusion of the survey would be more or less fair. However the survey doesn’t come even close.
Brandon Shollenberger, who is guest blogger at The Blackboard, was the first who reported that actually only 65 papers have been rated “category 1.” Yes that’s right, only 65 abstracts clearly “mention that human activity is a dominant influence or has caused most of recent climate change (>50%);” 65 on a total of 12,000 is 0.5%. So a completely fair conclusion from their survey is that only 1 in 200 abstracts explicitly mentioned that humans are dominating climate. If you ignore the 8000 papers that were labeled category 4 (neutral, meaning having no position on AGW) the 65 would be 1.6%. The paper reported that only 78 papers (1.9% if you ignore the 8000 neutral abstracts) rejected AGW.
Now where is the 97% endorsement of AGW coming from? What the authors did is to add up the numbers of categories 1 to 3 and of the categories 5 to 7 which I show below:
Category 1: 65 [Explicit endorsement with quantification]
Category 2: 934 [Explicit endorsement without quantification]
Category 3: 2933 [Implicit endorsement]
Category 4: 8261 [No position or uncertain]
Category 5: 53 [Implicit rejection]
Category 6: 15 [Explicit rejection without quantification]
Category 7: 10 [Explicit rejection with quantification]
Total: 12271
As you can see the 78 “rejection of AGW” abstracts are the added number of categories 5-7. Categories 1-3 together add up to 3932 papers. This 3932 divided by 4010 (the total of categories 1-3 + 5-7) gives their impressive 97% …However of these 3932 abstracts 2933 (75%) fall in category 3. Now how strong is the endorsement of AGW in this category? Here is the description:
3. Implicit Endorsement of AGW
3.1 Mitigation papers that examine GHG emission reduction or carbon sequestration, linking it to climate change.
3.2 Climate modeling papers that talks about emission scenarios and subsequent warming or other climate impacts from increased CO2 in the abstract implicitly endorse that GHGs cause warming.
3.3 Paleoclimate papers that link CO2 to climate change.
3.4 Papers about climate policy (specifically mitigation of GHG emissions) unless they restrict their focus to non-GHG issues like CFC emissions in which case neutral.
3.5 Modeling of increased CO2 effect on regional temperature – not explicitly saying global warming but implying warming from CO2.
3.6 Endorsement of IPCC findings is usually an implicit endorsement.
I like 3.2: “endorse that GHG’s cause warming.” I also strongly agree with this part of 3.5: “implying warming from CO2?. The meaningless result of their whole exercise is that 75% of the abstracts that say something about AGW at all “link CO2 to climate change” or “imply warming from CO2. [This does not necessarily mean endorsement or consensus.]
If we just say “Humans cause some global warming,” we could be supporting a value 20% or 90%. Despite being able to support either position, we’d land in the top categories. That means the results will automatically be skewed toward the top.
Australian science reporter Jo Nova has some other choice comments in her post, “Cook’s fallacy ‘97% consensus’ study is a marketing ploy some journalists will fall for.”
“What does a study of 20 years of abstracts tell us about the global climate? Nothing. But it says quite a lot about the way government funding influences the scientific process.”
Jo Nova cuts to the chase with the following points (which are further explained in her post):
1. Thousands of papers support man-made climate change, but not one found the evidence that matters.
2. Cook’s study shows 66% of papers didn’t endorse man-made global warming. Cook calls this “an overwhelming consensus”.
3. Cook’s method is a logical fallacy: Argument from Authority. This is not science, it’s PR.
4. The number of papers is a proxy for funding.
5. Most of these consensus papers assume the theory is correct but never checked. They are irrelevant.
Read her post for seven more points.
All of this is about consensus, as if consensus is equated with truth, but it isn’t, see my post “On Consensus in Science.” In that post I show the origins of previous claims of consensus and show several examples of famous errors of consensus.
“It does not matter who you are, or how smart you are, or what title you have, or how many of you there are, and certainly not how many papers your side has published, if your prediction is wrong then your hypothesis is wrong. Period.” -Richard Feynman
“I regard consensus science as an extremely pernicious development that ought to be stopped cold in its tracks. Historically, the claim of consensus has been the first refuge of scoundrels; it is a way to avoid debate by claiming that the matter is already settled. Whenever you hear the consensus of scientists agrees on something or other, reach for your wallet, because you’re being had.” -Michael Crichton.
wryheat 