<p>Astrophysicist Ken Rice, author of the "And Then There's Physics" blog on climate science, <a href="https://andthentheresphysics.wordpress.com/2017/08/22/machine-unlearning/" rel="noreferrer">dismisses this particular approach</a> for, among other things, failing to address the underlying physics of the climate system:</p>

<blockquote>
  <p>...if you are going to use something like machine learning to make
  predictions about the future, you do need to be pretty confident that
  the data that you use to train the machine learning algorithm presents
  a reasonable representation of the system you’re trying to model. This
  requires some actual understanding of the system being considered. If
  you change it by, for example, pumping lots of greenhouse gases into
  the atmosphere, then the training data will be almost certainly not be
  appropriate.</p>
</blockquote>

<p>Climate researcher Gavin Schmidt has <a href="https://twitter.com/ClimateOfGavin/status/900341454232371200" rel="noreferrer">also noted errors in figure 2</a>:</p>

<blockquote>
  <p>Their time axis is off by ~35 years and magnitude is too large by
  ~10%. So their '20th C' is actually 1845-1965.</p>
</blockquote>

<p>Which means that Abbot and Morahasy's analysis also doesn't include the end of the 20th century and the most recent decades of significant warming.</p>

CC BY-SA 3.0
2017-08-23T16:33:58.317Z
2017-08-23T16:33:58.317Z
jeffronicus
<p>Machine learning may sound like some magical phrase, that is able to solve every problem, but it's definetly not.
I don't have the paper available right now, but what I can say in general right now is the following:</p>

<p>What machine learning is good at, is finding patterns in any number of dimensions. If you apply machine learning to a time series (a 1-D data space) that fluctuates boringly around a mean average, as it did in pre-industrial times, then the only thing it will be able to predict is the continuation of this trend.</p>

<p>Furthermore, taking the ANN to predict data outside of it's training range is equivalent to just taking an extremely-high order polynomial through all your datapoints and then extrapolating into the future. The former is a classical example of data manipulation, also strongly advised against in every introductory physics or statistics class.</p>

<p>The last paragraph shall, without rigorous proof, just show that using ANN to predict trends in any kind of data, is not very smart and reveals a fundamental misunderstanding of how ANN work.</p>

CC BY-SA 3.0
2017-08-23T23:19:19.590Z
2017-08-23T23:19:19.590Z
AtmosphericPrisonEscape
<p>The paper is deeply flawed from both the climate science and machine learning perspectives.  The most obvious being the most eye-catching claim that equilibrium climate sensitivity is approximately 0.6C, which if true would overturn our understanding of the climate system.  However the paper doesn't actually explain how this figure of 0.6C is obtained from a "largest deviation" of 0.2C, it is basically just a hand-wave.  Also the largest deviation is not 0.2, this is the largest <em>average</em> (mean absolute) deviation seen in the proxies, and you can have a mean absolute deviation without there being a trend that you could relate to increased GHG concentrations and hence estimate ECS.  More importantly, this would give an estimate of <em>transient</em> climate sensitivity, not <em>equilibrium</em> climate sensitivity, and you can't reliably estimate ECS (which is global) from regional or sub-regional proxy records.  </p>

<p>Approaches such as the one taken in this paper, which seek to see how much of the data can be attributed to "climate cycles" with the remainder being taken as the anthropogenic component are inherently biased towards low estimates of ECS.  This is because of omitted variable bias; because the anthropogenic forcing signals are not included in the model, if the net effect of independent changes in the forcings is correlated with a sinusoidal component, they will be wrongly attributed to these climate cycles, when in fact they are produced by the forcings.  Models like this can only be used to estimate lower bounds on ECS, likewise if you make a model using the forcings as inputs and treat the residual as being "natural variability" it will tend to over-estimate ECS, giving an upper bound estimate.  The  Abbot and Morahasy paper cites a similar <a href="http://www.sciencedirect.com/science/article/pii/S0304380014000404" rel="noreferrer">paper</a> by Loehle, but sadly does not also cite the <a href="http://www.sciencedirect.com/science/article/pii/S0304380014004876" rel="noreferrer">comment paper</a> (of which I was the lead author, note the corregendum).  This is poor scholarship, and sadly  Abbot and Morahasy have gone on to make many of the same basic mistakes (but with a more complicated model), which is a shame.</p>

<p>Rather than using the original datasets (many of which are freely available) the authors chose to digitise images of the datasets instead.  This seems somewhat bizarre, and Gavin Schmidt <a href="https://twitter.com/ClimateOfGavin/status/900366059072389121" rel="noreferrer">points out via twitter</a> that in at least one case the dataset has not been scaled or aligned correctly (andends at 1965, and so does not include the recent warming where anthropogenic contributions are most evident).  It also transpires that Figures 5 and 9 are identical.</p>

<p>The paper says <em>"However, superior fitting to the temperature proxies are obtained by using the sine wave components and composite as input data. This was established by comparing the spectral analysis composite method versus the ANN method for the training periods."</em>.   Evaluating performance on the training data is a classic error in the use of machine learning that people used to make all the time in the late 1980s and 90s, but is rarely seen today.  If you have two nested models (one can be implemented as a special case of the other) of different complexities, then the more complex model will always have a lower training set error, if only because it has more capacity to memorize the random noise in the data, but that doesn't mean it is the more accurate model.  For that you need out-of-sample comparisons, which are absent from the paper.</p>

<p>There is no handling of uncertainty in the model (for instance the periods of the cyclic components are not known exactly, cycles with slightly different periodicities will explain the observations almost as well), and likewise there will be uncertainty in the parameters of the neural net, but none of this is propagated through to give the uncertainty in the estimate of ECS.  As seen in the comment on the Loehle paper, this can be substantial.</p>

<p>Table 13 seems to suggest that paleoclimate studies give lower ECS estimates than GCMs, which suggests a rather selective view of the paleoclimate studies, which generally indicate high ECS IIRC.</p>

<p>The study also has problems with too many degrees of researcher freedom (e.g. how was the particular subset of proxies chosen?) and there is a lot of (automated) exploration of model architectures and feature selection, which is often a recipe for over-fitting in model selection.  It is also not clear why the observation should be a <em>non-linear</em> function of the cyclic variables (especially given that the cycles were obtained from the data by linear analysis).</p>

CC BY-SA 3.0
2017-08-24T14:26:37.660Z
Why was this paper accepted if its so flawed?
DLV
CC BY-SA 3.0
2017-08-24T15:39:20.847Z
@DavidV You would have to ask the journal -- but you'll have to do it before January 2018, when GeoResJ will cease publication after four years, per its website.
jeffronicus
CC BY-SA 3.0
2017-08-25T07:09:16.927Z
@DavidV Reviewers are only human just like the rest of us and make mistakes, but from the Guide for Authors: "Please submit, with the manuscript, the names, addresses and e-mail addresses of four potential referees. Note that the editors retain the sole right to decide whether or not the suggested reviewers are used."  This is a recipe for pal-review, and is a feature of many journals that have published bad papers by climate skeptics.  Journals really shouldn't do this (https://academia.stackexchange.com/questions/10474/how-to-suggest-reviewers-for-a-journal/10483#10483)!
Dikran Marsupial
CC BY-SA 3.0
2017-08-25T07:10:04.490Z
@jeffronicus good way of making sure nobody can publish a comment paper there pointing out the flaws! ;o)
Dikran Marsupial
CC BY-SA 3.0
2017-08-24T08:44:35.227Z
2017-08-24T08:44:35.227Z
Dikran Marsupial
<p>Recently, a paper was published by <a href="https://doi.org/10.1016/j.grj.2017.08.001" rel="nofollow noreferrer">Abbot and Morahasy (2017)</a> - see specifically <a href="http://www.sciencedirect.com/science/article/pii/S2214242817300426#fig0002" rel="nofollow noreferrer">Fig. 2</a>. The thesis of the paper is to train an artificial neural network (ANN) on the temperature time series for pre-industrial ages. Then do forecasting with the ANN to predict the temperature time series in the 20th century. Because the ANN was trained on pre-industrial time series, then we can say that industrialization is irrelevant to global warming.</p>

<p>This goes completely contrary to the <a href="https://www.ipcc.ch/pdf/assessment-report/ar5/wg1/WG1AR5_TS_FINAL.pdf" rel="nofollow noreferrer">IPCC AR5 technical summary</a>, which shows graphs that global warming is due to anthropogenic causes. See page 74 of the latter report.</p>

<p>So which do you trust and why?</p>

CC BY-SA 3.0
2017-08-23T03:27:29.793Z
I dont know if you have access to the full article. One of the author has a blog post about it. http://jennifermarohasy.com/2017/08/recent-warming-natural/
DLV
CC BY-SA 3.0
2017-08-23T03:29:01.060Z
And theyre doing time series forecasting with an ANN. This is commonly done nowadays and with very complex data like financial time series...
DLV
CC BY-SA 3.0
2017-08-23T04:01:29.900Z
I have the full paper and could email it to anyone in need of it.
DLV
CC BY-SA 3.0
2017-08-23T04:21:32.260Z
This B. Macfie Family Foundation that funds the research is very iffy. (I'm just finding out)
DLV
CC BY-SA 3.0
2017-08-23T05:58:12.180Z
You can't do that because global warming isn't something determined from statistics.  It's instead a prediction from the known physical properties of CO2 &c.  See e.g. here https://history.aip.org/climate/index.htm for a fuller explanation.
jamesqf
CC BY-SA 3.0
2017-08-24T04:21:27.553Z
@rjzii: That rather assumes that there are natural cycles other than the ones we already know about, and have included in climate models.  But now that I think on it more, another major problem here is false positives.  The developers seem to expect to get a certain result - that AGW is really just a natural cycle - from their ANN.  Therefore, if the straightforward training doesn't show the result they want, they will accidentally or intentionally re-train it until it until it does.
jamesqf
CC BY-SA 3.0
2017-08-24T18:35:49.893Z
@David V:  That link has everything you need to know about the authors' bias in the second paragraph.  "...a speculative theory about the absorption and emission of infrared radiation by carbon dioxide..."  They think CO2 emission & absorption are SPECULATIVE, and not something both measured and computed from physical principles?  As well as being used in various devices, such as CO2 lasers.
jamesqf
CC BY-SA 3.0
2017-08-27T16:16:47.723Z
@rjzii note that "anthropomorphic" means [having human form or characteristics](https://en.oxforddictionaries.com/definition/anthropomorphic). That's the wrong word in this context. The word you want is "anthropogenic", which means [generated by human activities](https://en.oxforddictionaries.com/definition/anthropogenic)
410 gone
<climate-change>
Does this paper, using neural nets, prove industrialization is irrelevant to global warming?
DLV
# Does this paper, using neural nets, prove industrialization is irrelevant to global warming?

(Asked by: username_0 on 2017-08-23)

Recently, a paper was published by [Abbot and Morahasy (2017)](https://doi.org/10.1016/j.grj.2017.08.001) - see specifically [Fig. 2](http://www.sciencedirect.com/science/article/pii/S2214242817300426#fig0002). The thesis of the paper is to train an artificial neural network (ANN) on the temperature time series for pre-industrial ages. Then do forecasting with the ANN to predict the temperature time series in the 20th century. Because the ANN was trained on pre-industrial time series, then we can say that industrialization is irrelevant to global warming.

This goes completely contrary to the [IPCC AR5 technical summary](https://www.ipcc.ch/pdf/assessment-report/ar5/wg1/WG1AR5_TS_FINAL.pdf), which shows graphs that global warming is due to anthropogenic causes. See page 74 of the latter report.

So which do you trust and why?

**Tags:** climate-change

> **Comments:**
> **username_0**: I dont know if you have access to the full article. One of the author has a blog post about it. http://jennifermarohasy.com/2017/08/recent-warming-natural/  
> **username_0**: And theyre doing time series forecasting with an ANN. This is commonly done nowadays and with very complex data like financial time series...  
> **username_0**: I have the full paper and could email it to anyone in need of it.  
> **username_0**: This B. Macfie Family Foundation that funds the research is very iffy. (I'm just finding out)  
> **username_1**: You can't do that because global warming isn't something determined from statistics.  It's instead a prediction from the known physical properties of CO2 &c.  See e.g. here https://history.aip.org/climate/index.htm for a fuller explanation.  
> **username_1**: @rjzii: That rather assumes that there are natural cycles other than the ones we already know about, and have included in climate models.  But now that I think on it more, another major problem here is false positives.  The developers seem to expect to get a certain result - that AGW is really just a natural cycle - from their ANN.  Therefore, if the straightforward training doesn't show the result they want, they will accidentally or intentionally re-train it until it until it does.  
> **username_1**: @David V:  That link has everything you need to know about the authors' bias in the second paragraph.  "...a speculative theory about the absorption and emission of infrared radiation by carbon dioxide..."  They think CO2 emission & absorption are SPECULATIVE, and not something both measured and computed from physical principles?  As well as being used in various devices, such as CO2 lasers.  
> **username_2**: @rjzii note that "anthropomorphic" means [having human form or characteristics](https://en.oxforddictionaries.com/definition/anthropomorphic). That's the wrong word in this context. The word you want is "anthropogenic", which means [generated by human activities](https://en.oxforddictionaries.com/definition/anthropogenic)  

---

## 3 answers

### Answer by username_5 on 2017-08-24. Score: 19 | Accepted answer

The paper is deeply flawed from both the climate science and machine learning perspectives. The most obvious being the most eye-catching claim that equilibrium climate sensitivity is approximately 0.6C, which if true would overturn our understanding of the climate system. However the paper doesn't actually explain how this figure of 0.6C is obtained from a "largest deviation" of 0.2C, it is basically just a hand-wave. Also the largest deviation is not 0.2, this is the largest *average* (mean absolute) deviation seen in the proxies, and you can have a mean absolute deviation without there being a trend that you could relate to increased GHG concentrations and hence estimate ECS. More importantly, this would give an estimate of *transient* climate sensitivity, not *equilibrium* climate sensitivity, and you can't reliably estimate ECS (which is global) from regional or sub-regional proxy records.

Approaches such as the one taken in this paper, which seek to see how much of the data can be attributed to "climate cycles" with the remainder being taken as the anthropogenic component are inherently biased towards low estimates of ECS. This is because of omitted variable bias; because the anthropogenic forcing signals are not included in the model, if the net effect of independent changes in the forcings is correlated with a sinusoidal component, they will be wrongly attributed to these climate cycles, when in fact they are produced by the forcings. Models like this can only be used to estimate lower bounds on ECS, likewise if you make a model using the forcings as inputs and treat the residual as being "natural variability" it will tend to over-estimate ECS, giving an upper bound estimate. The Abbot and Morahasy paper cites a similar [paper](http://www.sciencedirect.com/science/article/pii/S0304380014000404) by Loehle, but sadly does not also cite the [comment paper](http://www.sciencedirect.com/science/article/pii/S0304380014004876) (of which I was the lead author, note the corregendum). This is poor scholarship, and sadly Abbot and Morahasy have username_2 on to make many of the same basic mistakes (but with a more complicated model), which is a shame.

Rather than using the original datasets (many of which are freely available) the authors chose to digitise images of the datasets instead. This seems somewhat bizarre, and Gavin Schmidt [points out via twitter](https://twitter.com/ClimateOfGavin/status/900366059072389121) that in at least one case the dataset has not been scaled or aligned correctly (andends at 1965, and so does not include the recent warming where anthropogenic contributions are most evident). It also transpires that Figures 5 and 9 are identical.

The paper says *"However, superior fitting to the temperature proxies are obtained by using the sine wave components and composite as input data. This was established by comparing the spectral analysis composite method versus the ANN method for the training periods."*. Evaluating performance on the training data is a classic error in the use of machine learning that people used to make all the time in the late 1980s and 90s, but is rarely seen today. If you have two nested models (one can be implemented as a special case of the other) of different complexities, then the more complex model will always have a lower training set error, if only because it has more capacity to memorize the random noise in the data, but that doesn't mean it is the more accurate model. For that you need out-of-sample comparisons, which are absent from the paper.

There is no handling of uncertainty in the model (for instance the periods of the cyclic components are not known exactly, cycles with slightly different periodicities will explain the observations almost as well), and likewise there will be uncertainty in the parameters of the neural net, but none of this is propagated through to give the uncertainty in the estimate of ECS. As seen in the comment on the Loehle paper, this can be substantial.

Table 13 seems to suggest that paleoclimate studies give lower ECS estimates than GCMs, which suggests a rather selective view of the paleoclimate studies, which generally indicate high ECS IIRC.

The study also has problems with too many degrees of researcher freedom (e.g. how was the particular subset of proxies chosen?) and there is a lot of (automated) exploration of model architectures and feature selection, which is often a recipe for over-fitting in model selection. It is also not clear why the observation should be a *non-linear* function of the cyclic variables (especially given that the cycles were obtained from the data by linear analysis).

> **Comments:**
> **username_0**: Why was this paper accepted if its so flawed?  
> **username_3**: @DavidV You would have to ask the journal -- but you'll have to do it before January 2018, when GeoResJ will cease publication after four years, per its website.  
> **username_5**: @DavidV Reviewers are only human just like the rest of us and make mistakes, but from the Guide for Authors: "Please submit, with the manuscript, the names, addresses and e-mail addresses of four potential referees. Note that the editors retain the sole right to decide whether or not the suggested reviewers are used."  This is a recipe for pal-review, and is a feature of many journals that have published bad papers by climate skeptics.  Journals really shouldn't do this (https://academia.stackexchange.com/questions/10474/how-to-suggest-reviewers-for-a-journal/10483#10483)!  
> **username_5**: @username_3 good way of making sure nobody can publish a comment paper there pointing out the flaws! ;o)  

---

### Answer by username_3 on 2017-08-23. Score: 16

Astrophysicist Ken Rice, author of the "And Then There's Physics" blog on climate science, [dismisses this particular approach](https://andthentheresphysics.wordpress.com/2017/08/22/machine-unlearning/) for, among other things, failing to address the underlying physics of the climate system:

> ...if you are going to use something like machine learning to make
> predictions about the future, you do need to be pretty confident that
> the data that you use to train the machine learning algorithm presents
> a reasonable representation of the system you’re trying to model. This
> requires some actual understanding of the system being considered. If
> you change it by, for example, pumping lots of greenhouse gases into
> the atmosphere, then the training data will be almost certainly not be
> appropriate.

Climate researcher Gavin Schmidt has [also noted errors in figure 2](https://twitter.com/ClimateOfGavin/status/900341454232371200):

> Their time axis is off by ~35 years and magnitude is too large by
> ~10%. So their '20th C' is actually 1845-1965.

Which means that Abbot and Morahasy's analysis also doesn't include the end of the 20th century and the most recent decades of significant warming.


---

### Answer by username_4 on 2017-08-23. Score: 2

Machine learning may sound like some magical phrase, that is able to solve every problem, but it's definetly not.
I don't have the paper available right now, but what I can say in general right now is the following:

What machine learning is good at, is finding patterns in any number of dimensions. If you apply machine learning to a time series (a 1-D data space) that fluctuates boringly around a mean average, as it did in pre-industrial times, then the only thing it will be able to predict is the continuation of this trend.

Furthermore, taking the ANN to predict data outside of it's training range is equivalent to just taking an extremely-high order polynomial through all your datapoints and then extrapolating into the future. The former is a classical example of data manipulation, also strongly advised against in every introductory physics or statistics class.

The last paragraph shall, without rigorous proof, just show that using ANN to predict trends in any kind of data, is not very smart and reveals a fundamental misunderstanding of how ANN work.


---

