Cook’s survey not only meaningless but also misleading

The new Cook et al. survey of the scientific literature is attracting worldwide media attention and even the American president tweeted (“Ninety-seven percent of scientists agree: #climate change is real, man-made and dangerous.”) about it. Yesterday my own first reaction on twitter was: “All this talking about consensus http://iopscience.iop.org/1748-9326/8/2/024024/pdf/1748-9326_8_2_024024.pdf … doesn’t improve our understanding of climate even a tiny bit” And I added that “it shows again how meaningless this kind of consensus analysis is”.

Now I still support the label “meaningless” but today I will add “misleading” to the list.

So what’s all the fuzz about? Cook et al. selected around 12,000 scientific abstracts that contained the words “global warming” or “global climate change” published in the period 1991-2011. With a large group of volunteers they then rated the papers using 7 categories. Around 8000 of the abstracts (2/3) take no clear position on Anthropogenic Global Warming (AGW). Of the remaining ~4000 abstracts more than 97% “endorse AGW” according to the paper. Only a tiny amount (78 papers) “reject AGW”. Hence they claim again that there is a consensus, that the debate is over and also that there is a gap between scientists and the public (see graph above). A much larger percentage of the scientists “endorses AGW” than the public at large.

Misleading
Now here comes the misleading part. If an abstract/paper “endorses AGW”, what would this mean for most people? Let’s look again at the tweet of Obama: “#climate change is real, man-made and dangerous”. If this is what it means for the president of the US, it probably means the same for many citizens who heard the news in the media. However, can this be sustantiated by the survey results? In no way.

To the credit of the researchers they made all their results available in a searchable database. Their rating system is online as well. There are 7 levels of endorsement, going from quantified endorsement of AGW all the way down to a quantified rejection of AGW. Seems fair enough. But here  is the issue. Only the first category can be regarded as a real or strong endorsement of AGW. Here is the description of category 1:

1. Explicit Endorsement of AGW with quantification
1.1 Mention that human activity is a dominant influence or has caused most of recent climate change (>50%).
1.2 Endorsing the IPCC without explicitly quantifying doesnt count as explicit endorsement – that would be implicit.

Now specifically look at 1.1. This comes close to the iconic statement from the IPCC AR4 report which said that “Most of the observed increase in global average temperatures since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations.” Now if 97% of the abstracts would repeat in slightly varying terms this major conclusion, than at least the conclusion of the survey would be more or less fair. However the survey doesn’t come even close.

Brandon Shollenberger, who is guest blogger at The Blackboard, was the first who reported that actually only 65 papers have been rated “category 1”. Yes that’s right, only 65 abstracts clearly “mention that human activity is a dominant influence or has caused most of recent climate change (>50%)”. 65 on a total of 12,000 is 0.5%. So a completely fair conclusion from their survey is that only 1 in 200 abstracts explicitly mentioned that humans are dominating climate. If you ignore the 8000 papers that were labelled category 4 (neutral, meaning having no position on AGW) the 65 would be 1.6%. The paper reported that only 78 papers (1.9% if you ignore the 8000 neutral abstracts) rejected AGW. (to be fair, as you can see in the table below only 10 papers fall in category 7 and therefore (7.1) “explicitly reject or minimise anthropogenic warming with a specific figure”.)

Now where is the 97% endorsement of AGW coming from? What the authors did is to add up the numbers of category 1 to 3 and of the category 5 to 7 which I show below*:

Category 1: 65
Category 2: 934
Category 3: 2933
Category 4: 8261
Category 5: 53
Category 6: 15
Category 7: 10
Total:         12271

As you can see the 78 “rejection of AGW” abstracts are the added number of category 5-7. Category 1-3 together adds up to 3932 papers. This 3932 divided by 4010 (the total of category 1-3 + 5-7) gives their impressive 97% (with the above numbers it is even 98%, see * for more explanation about how these numbers were found). However of these 3932 abstracts 2933 (75%) fall in category 3. Now how strong is the endorsement of AGW in this category? Here is the description:

3. Implicit Endorsement of AGW
3.1 Mitigation papers that examine GHG emission reduction or carbon sequestration, linking it to climate change
3.2 Climate modelling papers that talks about emission scenarios and subsequent warming or other climate impacts from increased CO2 in the abstract implicitly endorse that GHGs cause warming
3.3 Paleoclimate papers that link CO2 to climate change
3.4 Papers about climate policy (specifically mitigation of GHG emissions) unless they restrict their focus to non-GHG issues like CFC emissions in which case neutral
3.5 Modelling of increased CO2 effect on regional temperature – not explicitly saying global warming but implying warming from CO2
3.6 Endorsement of IPCC findings is usually an implicit endorsement. (updated this so it is more than just reference to IPCC but actual endorsement of IPCC)

I like 3.2: “endorse that GHG’s cause warming”. I also strongly agree with this part of 3.5: “implying warming from CO2”. The meaningless result of their whole exercise is that 75% of the abstracts that say something about AGW at all “link CO2 to climate change” or “imply warming from CO2″.

The misleading part is that they didn’t specify this result in their paper. Nowhere in their paper or in the supplementary material they even mentioned the total numbers in the different categories like I did in the simple table above. They only showed the total of category 1-3 in their figure 1(a):

 

Even the other >24% of AGW endorsement (based on the 4010) in category 2 is pretty meaningless:

2. Explicit Endorsement of AGW without quantification
2.1 Mention of anthropogenic global warming or anthropogenic climate change as a given fact.
2.2 Mention of increased CO2 leading to higher temperatures without including anthropogenic or reference to human influence/activity relegates to implicit endorsement.

In a comment under his post Shollenberger nicely explains who the different categories skew the result towards “endorsement of AGW”:

To give an example, if we say, “Humans are responsible for 40% of global warming,” that puts us in the bottom category. Change the number to 60%, and suddenly we’re in the top category. But what if we don’t give a number at all? If we just say “Humans cause some global warming,” we could be supporting a value 20% or 90%. Despite being able to support either position, we’d land in the top categories. That means the results will automatically be skewed toward the top.

 

Self confirming
Now for anyone who reads climate papers frequently this is totally obvious. Climate scientists have to frame their research in the abstract and there wouldn’t be so much climate papers if there was no concern for CO2.

So the whole result of this survey is completely self confirming. Because there is a concern for CO2 there is a lot of funding of climate science. This then generates a lot of climate science papers (they surveyed 12,000 but mention there are many more). In the abstracts scientists refer to the concern about CO2. The abstract then falls into category 2 or 3 and therefore almost all the papers “endorse AGW”.

 

* This is how you can replicate the numbers. Go to http://www.skepticalscience.com/tcp.php?t=search and type just a single letter in the search term box (I used “a”). This generates the total of 12,271 abstracts. Then select the whole period 1991-2011 and just search the 7 different categories. This generates the table from the blog post above.

 

Door |2013-05-17T13:21:17+00:0017 mei 2013|Analyse, Nieuws, opinie, Peer reviewed|37 Reacties

37
Reageer op dit artikel

avatar
37 Comment threads
0 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
8 Comment authors
RomanDave BurtonRZFerdinand EngelbeenHonest Broker Recent comment authors

Deze website gebruikt Akismet om spam te verminderen. Bekijk hoe je reactie-gegevens worden verwerkt.

  Subscribe  
nieuwste oudste meest gestemd
Abonneren op
TINSTAAFL
Gast
TINSTAAFL

Ach ja die Cook, zou jij een gebruikte auto van hem kopen? Obama’s geloofwaardigheid, die de laatste weken al zo’n dreun heeft gekregen, wordt hier zeker niet groter van. Die zit alleen maar de DEM AGW consensus te papagaaien.

Verbaast me overigens niets, Cook werkte samen met die andere charlatan Lewandowski. Dit soort misleidende, partijdige pseudo-wetenschap is zo typerend voor de huidige klimaatwetenschap. Ik zou me de ogen uit mijn hoofd schamen als ik daartoe behoorde.

trackback

[…] Cook’s survey not only meaningless but also misleading. Marcel Crok. […]

trackback

[…] Cook’s survey not only meaningless but also misleading. Marcel Crok. […]

Ferdinand Engelbeen
Gast

As an extra to this comment: the survey only looked at the abstracts. I have read several papers in the past that did look at natural influences on climate, with as conclusion that the natural influences were underestimated (and thus the influence of CO2 overestimated). Despite that, the abstract contained some sentences of adherence to the “concensus” about the influence of CO2. Probably because without such adherence… no publication. The same problem did occur with the previous Oreskes paper.

Even without the biases of the authors, if not looking beyond the abstracts, such study has very little value.

Bart Verheggen
Gast

Using the numbers from above, basically the following apples-to-apples fractions become apparent:

Of those abstracts making a statement about the quantitative contribution of human activity to the warming, 87% (65/75) endorsed dominant human causation.

Of those abstracts expressing an explicit position on the cause of global warming, 97.6% (999/1024) endorsed human causation.

In addition to the paper’s headline result:

Of those abstracts expressing a position on the cause of global warming, 97% (implicitly or explicitly) endorsed human causation of this warming.

And of course there’s a variety of apples-to-oranges comparisons, of which Marcel gives some examples.

http://ourchangingclimate.wordpress.com/2013/05/17/consensus-behind-the-numbers/

http://klimaatverandering.wordpress.com/2013/05/17/achter-de-getallen-van-de-consensus/

trackback

[…] Marcel Crok: “Cook’s survey not only meaningless but also misleading” https://www.destaatvanhet-klimaat.nl/2013/05/17/cooks-survey-not-only-meaningless-but-also-misleading/ […]

Ferdinand Engelbeen
Gast

Bart, Only categories 1 and 7 from Cook are quantitative, the other categories are qualitative only. So your sentence: Of those abstracts expressing a position on the cause of global warming, 97% (implicitly or explicitly) endorsed human causation of this warming. is wrong: 97% (implicitly or explicitly) endorsed human causation of some of this warming. Most may endorse implicitely or explicitely the IPCC/climate models range as base, but even that is problematic because the huge 1:3 range of the model projections for 2xCO2. Thus except for category 1, any other abstract not attributing anything explicitely, may implicitely take an effect… Lees verder »

trackback

[…] LINK […]

plazaeme
Gast

I thought Cook’s work was such a nonsense it wasn’t worth talking about it. Even thinking about it.

I was wrong. It is a broad red line, a case like Mann’s hockey stick, or climategate. Its main value is to show the gymnastic arguments of those who endorse it, and their “scientific” attitude. In this sense, it is priceless.

Thanks to Shollenberger and Marcel Crok. Very well exposed.

Honest Broker
Gast
Honest Broker

Climate scientists know these type of studies are generally a bunch of non-sense (just ask Hans von Storch & Dennis Bray or Roger Pielke Sr. & James Annan). From the ClimateGate emails Tom Wigley to Phil Jones on the work of Naomi Orekeskes: Science is no democracy. ### date: Thu, 12 Nov 2009 14:16:40 -0700 from: Tom Wigley subject: Re: [Fwd: Your Submission] to: Phil Jones Phil, This is weird. I used Web of Knowledge, “create citation report”, and added 1999 thru 2009 numbers. Can’t do you becoz of the too many PDJs problem. Here are 3 results … Kevin… Lees verder »

trackback

[…] That question has been discussed on many blogs.  The best explanation I’ve found is one by Dutch researcher Marcel Crok, “Cook’s survey not only meaningless but also misleading.“ […]

trackback

[…] That question has been discussed on many blogs.  The best explanation I’ve found is one by Dutch researcher Marcel Crok, “Cook’s survey not only meaningless but also misleading.“ […]

trackback

[…] That question has been discussed on many blogs.  The best explanation I’ve found is one by Dutch researcher Marcel Crok, “Cook’s survey not only meaningless but also misleading.“ […]

trackback

[…] That question has been discussed on many blogs.  The best explanation I’ve found is one by Dutch researcher Marcel Crok, “Cook’s survey not only meaningless but also misleading.“ […]

trackback

[…] That question has been discussed on many blogs.  The best explanation I’ve found is one by Dutch researcher Marcel Crok, “Cook’s survey not only meaningless but also misleading.“ […]

trackback

[…] That question has been discussed on many blogs.  The best explanation I’ve found is one by Dutch researcher Marcel Crok, “Cook’s survey not only meaningless but also misleading.“ […]

trackback

[…] That question has been discussed on many blogs.  The best explanation I’ve found is one by Dutch researcher Marcel Crok, “Cook’s survey not only meaningless but also misleading.“ […]

Ferdinand Engelbeen
Gast

As an extra: The figures for the authors who don’t give a position in the abstract of their paper, but give their opinion in the second round is very interesting: Among self-rated papers not expressing a position on AGW in the abstract, 53.8% were self-rated as endorsing the consensus. That gives a complete different view on the 60% papers which don’t give an opinion in their abstracts. If we may use that figure for all of the 60%, then we have complete different ratio’s of endorsement for AGW: Within the margins of error of the search (which still are huge):… Lees verder »

Ferdinand Engelbeen
Gast

The rest of the papers doesn’t have no position

typo…

The rest of the papers doesn’t have a position

trackback

[…] im Klimageschehen der Erde als wenig bedeutsam ansehen als umgekehrt. In dieser Studie hatte man, wie der niederländische Blogger Marcel Crok ausführt,   klimawissenschaftliche Veröffentlichungen erst klassifiziert und dann nach ihrer Aussagen zur […]

trackback

[…] was Dutch chemist and science journalist Marcel Crok, who points out many of the problems — here — concerning the Cook study’s misleading selection, categorizations, and descriptions of […]

RZ
Gast
RZ

Again and again the two main issues of the debate are mixed up for poltical purposes,

The risk-analysis (which dominates the public debate) and the scientific analysis are mixed up

the fact again demonstrates the threat to the academic climate, where this sort of rubbish is accepted in journals

trackback

[…] and Marcel Crok give a pretty good analysis of just how misleading Cook’s claims […]

trackback

[…] That question has been discussed on many blogs.  The best explanation I’ve found is one by Dutch researcher Marcel Crok, “Cook’s survey not only meaningless but also misleading.“ […]

Dave Burton
Gast

President Obama said, “Ninety-seven percent of scientists agree: #climate change is real, man-made and dangerous.”

It’s not true. He should have said, “97% of 3% of scientists think climate change is real, man-made & dangerous; the rest are divided.”

See: http://tinyurl.com/Clim97pct ( tinyurl[dot]com/Clim97pct )

trackback

[…] let’s examine the data behind the 97% figure. They rated the papers into 7 […]

trackback

[…] profess that 97% of climatologists believe in AGW. That’s a big number, but one based on a study that is grossly misleading. Of the 12,271 peer-reviewed articles included in the study, only 65 explicitly endorsed AGW […]

trackback

[…] was Dutch chemist and science journalist Marcel Crok, who points out many of the problems — here — concerning the Cook study’s misleading selection, categorizations, and descriptions of […]

trackback

[…] was Dutch chemist and science journalist Marcel Crok, who points out many of the problems — here — concerning the Cook study’s misleading selection, categorizations, and descriptions of […]

Roman
Gast

I take pleasure in, lead to I discovered exactly what I
used to be taking a look for. You’ve ended my four day
long hunt! God Bless you man. Have a nice day. Bye

trackback

[…] That question has been discussed on many blogs.  The best explanation I’ve found is one by Dutch researcher Marcel Crok, “Cook’s survey not only meaningless but also misleading.“ […]

trackback

[…] Naomi Oreskes was een van de eersten. John Cook et al kwamen later. Zij telden wetenschappelijke publicaties, waarin – naar hun oordeel – steun voor de AGW–hypothese tot uitdrukking werd gebracht. Dat is natuurlijk iets anders dan wetenschappelijk bewijs, want dat ontbreekt nog steeds. John Cook et al kwamen op 97%. De kritiek op de door hen gevolgde methode en de uitvoering van hun onderzoek was niet mals. Zie voor de filering van Cook et al onder meer Marcel Crok hier. […]

trackback

[…] Believe In Global Warming is TOTALLY BOGUS! (May 21, 2014) The State of the Climate – Cook’s survey not only meaningless but also misleading (May 17, 2013) WUWT – The Collapsing ‘Consensus’ (May 22, 2013) WUWT – Self […]

trackback

[…] Scientists Believe In Global Warming is TOTALLY BOGUS! (May 21, 2014) The State of the Climate – Cook’s survey not only meaningless but also misleading(May 17, 2013) WUWT – The Collapsing ‘Consensus’ (May 22, 2013) WUWT – Self admitted cyber […]