All the written submissions to the UK Energy and Climate Change Committee are now online. Many interesting things to read. Lots of critical submissions. Judith Curry calls Nic Lewis’ contribution a “tour de force” which it really is. Some readers here might know that Nic and I have been working for the past few months on a report for the Global Warming Policy Foundation about how AR5 dealt with climate sensitivity. His submission is a nice introduction/summary of our report which hopefully will be published in January. Below is Nic’s submission. I strongly encourage readers to read it in full.
Written evidence submitted by Nicholas Lewis
Credentials and statement of interests
I am an independent, self-funded climate science researcher. In recent years I have specialised in the key area of climate sensitivity. My work has been published in the peer reviewed literature and is cited and discussed in the IPCC’s Fifth Assessment Report (AR5). I was an expert reviewer of AR5.
Introduction and summary
- The terms of reference for this inquiry ask various questions. I address the following questions; my related conclusions are italicised.
- How robust are the conclusions in the AR5 Physical Science Basis report (AR5-WG1)?
In the central area of climate sensitivity, they are misleading. The substantial divergence between sensitivity estimates from, on the one hand, satisfactory studies based on instrumental observations over an extended period and, on the other hand, from flawed studies and from computer models was not brought out.
- Does the AR5 address the reliability of climate models?
Not adequately. Shorter-term warming projections by climate models have been scaled down by 40% in AR5, recognising that they are unrealistically high. But, inconsistently, no reduction has been made in longer term projections.
- Do the AR5 Physical Science Basis report’s conclusions strengthen or weaken the economic case for action to prevent dangerous climate change?
Although the conclusions fail to say so, the evidence in AR5-WG1 weakens the case since it indicates the climate system is less sensitive to greenhouse gases than previously thought.
- Climate sensitivity is a measure of how much the climate system warms each time the concentration of carbon dioxide in the atmosphere doubles. There are two principal measures used. Equilibrium climate sensitivity (ECS) is the amount of warming once the world ocean has fully warmed up, a process that takes more than a thousand years. ECS is believed to be a fairly stable property of the climate system, but is difficult to estimate accurately.Transient climate response (TCR) measures how much warming will take place over a 70-year period during which the carbon dioxide concentration doubles. TCR is easier to estimate than ECS, but may be less stable, and although more relevant to warming towards the end of the century is less useful than ECS when projecting over a wide range of timescales.
- Climate sensitivity is of direct policy relevance since it, and the level of uncertainty as to its value, is a key input into the economic models which drive cost-benefit analyses.
- Estimates of climate sensitivity have traditionally come from very complex computer models (GCMs) that simulate the global climate system. Attempts have also been made to calculate estimates based on paleoclimate data. However, in recent years it has become possible to calculate more robust values from instrumental observations of the climate, including surface and ocean temperature records and satellite data.
- The IPCC considers all observational ECS estimates in AR5 WG1. It concludes that estimates based on
- paleoclimate data reflecting different past climate states
- climate response to volcanic eruptions or solar changes
- satellite measurements of short-term changes in heat radiation are unreliable and/or unable to provide usefully well-constrained estimates. I agree with this conclusion. That leaves in essence only estimates based on observations of warming over multi-decadal periods. Useful surface temperature records extend back approximately 150 years (the ‘instrumental period’). However, global warming ‘in the pipeline’ is predominantly reflected in ocean heat uptake, calculated from changes in sub-surface temperatures, records ofwhich extend back only 50 years.
- There are two principal issues with the IPCC’s handling of the climate sensitivity area. Firstly the inclusion of sensitivity estimates from flawed observational studies that used unsuitable data, were poorly designed and/oremployed inappropriate statistical methodology. That obscured what should have been a key message from AR5 – that the best observational evidence now points to the climate system being substantially less sensitive to greenhouse gases than previously thought. Secondly, the elevation of computer models over observational evidence. Virtually all the projections of future climate change in AR5 are based on simulations by GCMs despite these being out of line with the best observational evidence.
- In particular, between since the Fourth Assessment Report of 2007 (AR4) and AR5, there has been a major reduction in the IPCC’s best estimate of how strong the effect – the ‘forcing’ – of atmospheric pollution (aerosols) is. This reduction results from improved understanding and incorporation of observations from satellite-based instruments; in AR4 the estimate was based primarily on GCMs. The reduction necessarily means that estimates of climate sensitivity should be lower too.
Poor experimental design and unsuitable data
- Most of the observational ECS estimates based on instrumental-period warming that were cited in AR5 used values for aerosol forcing that either:
- were consistent with the AR4 estimate; as noted above, this has now been superseded by the substantially lower estimate given in AR5
- directly reflected aerosol forcing levels in particular GCMs, that were substantially higher than the best estimates given in AR5
- were estimated from global mean temperature data. This last approach represents poor design. It is impossible to estimate aerosol forcing – which largely affects the northern hemisphere –with any accuracy without separate temperature data for, at a minimum, the northern and southernhemispheres.
- Some studies used GCM-derived estimates of anthropogenic warming or recent ocean heat uptake. Even where these also took observational data into account, it is unlikely that they fully did so. At least one study used its temperature data poorly.
Inappropriate statistical methodology
- Most of the observationally-based estimates of climate sensitivity explicitly adopt a ‘Bayesian’ statistical approach. A Bayesian approach demands that the researcher set out in mathematical terms a starting position for the value of the property of interest  – in this case the climate sensitivity. This ‘prior’ is then combined with the data to give a final result – the ‘posterior’. If the data is good quality then the final result will be little affected by the prior. But when data contains a weaker ‘message’ – as when estimating climate sensitivity – the choice of prior can greatly influence the final answer, and therefore be very contentious.
- Bayesian statistics is split into two schools: Subjective Bayesians and Objective Bayesians.
- Subjective Bayesians believe that the prior should only reflect existing knowledge about properties being estimated – in this case climate sensitivity. Moreover they take the view that all probabilities are subjective: they are personal and reflect the belief of the individual concerned. This is important – estimates arrived at using Subjective Bayesian methods – like those used in many IPCC estimates of climate sensitivity – are personal to a single decision maker: the investigator himself. As the Bayesian statistician Dennis Lindley wrote: ‘Uncertainty is a personal matter; it is not the uncertainty but your uncertainty’. The expert who provided the Subjective Bayesian statistical method used for UKCP09, the official UK climate projections, is crystal clear about that being the case: he talks about his climate, not the climate. The unsatisfactory nature of a Subjective Bayesian approach for scientific studies and policy decisions is self-evident.
- Objective Bayesians on the other hand try to derive mathematically a prior with minimal influence on the results; one that will best ‘let the data speak for themselves’ and thus give objective probabilities. The objective Bayesian approach is more suited to use in science, where it is standard for published results to reflect only the observational data from the experiment concerned.
- However, almost all of the Bayesian climate sensitivity estimates cited by the IPCC use a Subjective Bayesian approach. The starting position of many of them – their prior – is that all climate sensitivities are, over a very wide range, equally likely. In Bayesian terminology, they start from a ‘uniform prior’ in ECS. All climate sensitivity estimates shown in the 2007 IPCC AR4 report were stated  to be on a uniform prior in ECS basis. So are many cited in AR5.
- Use of uniform priors in ECS biases estimates upwards, usually substantially. The largest effect of uniform priors is on the upper uncertainty bounds for ECS, which are greatly inflated, making high future warming seem more plausible.
- Instead of uniform priors in ECS, some climate sensitivity estimates use ‘expert priors’. These are mainly representations of ‘consensus’ views of climate sensitivity, which largely reflect estimates of ECS derived from GCMs.Studies using expert priors typically produce ECS estimates that primarily reflect the prior, with the observational data having limited influence.
- To reiterate, findings using either uniform or expert priors for estimating ECS are subjective: they do not objectively reflect the data. The use of the Subjective Bayesian approach is inappropriate both in science and for informing the policy process, where objectivity must be paramount.
- The inquiries into the Climategate affair noted the lack of interaction between climatologists and professional statisticians, which may explain the ubiquity of such inappropriate methodology in this area. Steve Jewson, one of the few statistical experts involved in climate science to have compared objective and subjective Bayesian methods of estimating climate sensitivity, has commented as follows (and I quote his headings):
- The results from uniform priors are arbitrary and hence non-scientific
- If you use a uniform prior for [ECS], someone might accuse you of choosing the prior to give high rates of climate change
- The results may well be nonsense mathematically
- You risk criticism from more or less the entire statistics community
- You risk criticism from scientists in many other disciplines too
- If your paper is cited in the IPCC report, IPCC may end up losing credibility
- There is a perfectly good alternative, that solves all these problems 
- The alternative that Jewson mentions, the Objective Bayesian approach, was until recently almost unheard of in climate sensitivity studies.
- In AR4, the results of the high-quality Forster and Gregory 2006 empirical study were restated by the IPCC on the uniform prior in ECS basis – the one that Jewson and I have both noted is unscientific. AR4 stated that the highest tenable value of ECS found by Forster and Gregory was 14.2°C.  However, in AR5, the value is shown to be just 3.5°C when calculated on the objective basis used in the original Forster and Gregory paper. 
- Despite this, many studies cited in AR5 were still prepared using uniform priors in ECS. This biases all of the affected estimates towards higher warming. I made this point in my review comments on the draft AR5 report.
- Several of the studies cited in AR5 used so-called ‘expert priors’. As noted above, their ECS estimates to a large extent reflected the priors used rather than the data and are therefore unscientific and unreliable.
- In the area of climate sensitivity then, the IPCC includes many studies that are severely flawed – as regards statistical methodology and/or their design or data used – and therefore provide scientifically unsound estimates. 
Climate models versus empirical observation
- In AR4 the IPCC gave the likely range for ECS as 2–4.5°C  with a ‘best estimate’ of 3°C. These figures were unchanged from the previous report save for a slight increase in the lower bound of the range, from 1.5°C to 2.0°C.
- These estimates largely reflected the estimates derived from GCMs.  The newer (CMIP5) GCMs used in AR5 have almost the same ECS and TCR ranges and averages as those used in AR4. The main influence thatestimates based on empirical observations seem to have had on the AR5 report is a reduction of the lower ‘likely’ bound for ECS back down to 1.5°C and slight increases in the assessed probabilities that TCR is below 1°C or3°C.
- However, as noted above, between AR4 and AR5 there has been a major reduction in the IPCC’s estimate of the cooling strength of aerosol pollution, which necessarily implies that estimates of climate sensitivity should besubstantially lower than previous estimates. However, the current CMIP5 generation of GCMs still very largely reflects the earlier understanding of a stronger aerosol effect.
- A particularly robust way of empirically estimating climate sensitivity is the so-called ‘energy-budget’ method, which is based on a fundamental physical law – the conservation of energy.  Energy-budget best estimates  of ECS fall in a range between 1.5°C and 2.0°C (1.25–1.4°C for TCR), depending on the exact periods chosen for analysis. Using the longest available periods that were free of major volcanism gives a ECS best estimate of approximately 1.7°C (1.3°C for TCR). 
- Since AR4 a series of papers have derived estimates of ECS and TCR from observational data. Although not done in AR5, if those estimates:
- using methods considered in AR5 to provide unreliable or ill-constrained estimates;
- involving poor experimental design or unsuitable data; and/or
- that were significantly affected by use of subjective priors (uniform or ‘expert’) are eliminated from the assessment, the average ECS for those that remain  is, at 1.8°C, considerably lower than the GCM average of 3.2°C, but close to the energy-budget best estimate of 1.7°C that was outlined in the last paragraph.
- For the shorter-term TCR measure, the discrepancy between the average of the best observational estimates is, at 1.4°C,  also substantially lower than the average of the GCM value of 1.8°C, but close to the energy budget estimate based on the best AR5 data of 1.3°C.
- The discrepancy between observational and GCM-based estimates of climate sensitivity is stark and of huge importance to policymakers. In my review comments on the draft AR5 report, I said that the report should give separate ECS ranges for the estimates derived from observational evidence and those from GCMs so that this important information was made clear. Unfortunately, the IPCC authors chose not to do this, so avoiding the need to discuss the substantial difference between them. Instead AR5 reduced the lower bound of its ECS estimate to 1.5°C and gave no best estimate.
- There are good reasons for observational estimates of climate sensitivity to be preferred to GCM estimates in policy decisions. Although in some respects the GCMs reflect basic physics, many aspects of the climate system are‘parameterised’ – approximate representations are used. The climate processes involved are often poorly understood – climate science is immature – and the ways they are modelled embody unproven hypotheses and uncertainestimates of key input parameters. Some of those poorly understood processes, such as those controlling cloud behaviour, are critical determinants of GCM estimates of ECS and TCR.
- The output of the models is therefore inherently doubtful, and these doubts have been borne out in practice. The failure of the computer models to predict the hiatus in rising temperatures over the last 15 years has been widely reported. AR5 attributes this to several causes, and points out that internal climate variability is substantial over 15 years. However, as Figure 1 shows, over a climatically-relevant 35 year period virtually all of the CMIP5 GCMs predict (grey boxes) significantly more warming than has actually been observed (red line).
- Apart from these known shortcomings in GCMs, it is fundamental to the scientific method that when modelled values do not agree with observations then the hypothesis embodied in the model is modified or rejected. The refusal in AR5 to accept the implications of the best observational evidence and of the over-estimation of warming by the climate models and accordingly to either:
- reject the ensemble of GCM projections;
- use projections from a subset of GCMs with ECS and TCR values fairly close to the best observational estimates; or
- scale all GCM projections to reflect those estimates is unscientific.
- Note that the CMIP5 GCMs give an estimate for the warming over the next two decades as 0.48–1.15°C.  In the AR5-WG1 final draft, however, that estimate was reduced by 40% to 0.3–0.7°C, apparently recognising that overall the models were warming unrealistically quickly. Inconsistently, no change was made to the longer term GCM projections.
Estimates of future warming
- Warming over the rest of this century is related mainly to the 70-year TCR measure rather than the longer-term ECS measure. As noted above, the best observational evidence suggests a TCR of 1.3–1.4°C compared to the1.8°C average of the GCMs. In fact, the estimates of future warming produced by the models are on average 10–20% higher than their TCR values imply.
- Table 1 shows, for various scenarios of future greenhouse gas concentration pathways, estimates of global surface temperatures towards the end of the 21st century based on GCMs and alternatively based on a best observational estimate of TCR. Note that the mean global surface temperature over the decade 2003–2012 was 0.2°C higher than that for the 1986–2005 baseline used, following AR5, for Table 1. Therefore, projected warming measured from today is 0.2°C lower than shown in Table 1.
- If TCR really is 1.35°C then under RCP8.5 – the worst-case, business-as-usual scenario – the end of the 21st century will be approximately 2°C warmer than today.
- The meta-analysis in Tol (2009) , of fourteen estimates from economists, suggests that a temperature of 2°C warmer than today is likely to have a negligible impact on welfare.
 Observationally-based methods involve some limited use of models.
 In the form of a weight given to each possible value for it.
 The posterior takes the form of a probability distribution for the property of interest rather than a point estimate.
 Such a prior is not a representation of an understanding of the likely value of what is being estimated, but is simply a prior calculated to give an objective answer.
 Incorrectly in two cases.
 When, as is very much the case for ECS, the parameter involved does not have a straight line relationship with the observational data from which it is being estimated, a uniform prior generally prevents the estimate fairly reflecting the data.
 Using a mathematical, noninformative, prior.
 The top of the 5-95% range for the study’s ECS estimate as given in Table 9.3 of AR4-WG1.
 WG1 report, Figure 10.20.b).
 Forster and Gregory 2006 used standard non-Bayesian statistics, which are intrinsically objective and (as often the case) gave the same results as using an objective Bayesian method.
A critical analysis that I wrote of Instrumental and Combination ECS estimates shown in Figure 10.20.b) of AR5-WG1 is available at: http://niclewis.wordpress.com/ipcc-ar5-climate-sensitivity-and-other-issues/ar5_ecs_estimates/. Likewise an analysis of TCR estimates shown in Figure 10.20.a) of AR5-WG1 is available at http://niclewis.wordpress.com/ipcc-ar5-climate-sensitivity-and-other-issues/ar5_tcr_estimates/
 17–83% probability
 In other words, the average of the GCM estimates and their range (which was interpreted conservatively).
 In principle, studies that used observational data to form their own estimate of the aerosol effect would be unaffected by a revision in the consensus estimate of it. However, in practice all such climate sensitivity studies featured in AR4 were badly flawed in other ways.
 The method involves comparing changes in the average levels of three key variables between two multi-year periods, normally one early in the instrumental period and one at the end of it.
 Based on (a) the AR5 best estimates of the combined warming/cooling strength (forcing) of greenhouse gases, aerosols and other agents over the instrumental period, and of ocean heat uptake; (b) using the longest principal global surface temperature dataset; and (c) measuring changes between multi-year periods early in and at the end of the instrumental period with similar volcanic activity.
 . References herein to best estimates are to medians (50% probability points) unless otherwise stated.
 Aldrin et al. (2012) (the estimate of which was, exceptionally, not significantly affected by the subjective Bayesian method used); Ring et al. (2012) (average of four estimates); Otto et al. (2013) (averaging the 2000-2009 and 1970-2009 period estimates); and Lewis (2013) (preferred main results estimate).
 Taking median estimates from Schwartz (2012), Gillett et al. (2013) and (averaging the 2000-2009 and 1970-2009 period estimates) Otto et al. (2013). Aldrin et al. (2012) and Lewis (2013) did not provide ranges for TCR, but their ECS estimates imply TCR best estimates below 1.4°C.
 2016–2035 relative to 1986-2005
 Tol, R. 2009. The Economic Effects of Climate Change. Journal of Economic Perspectives, 23(2): 29-51.