<p>Model data seta can vary in several ways from a reference data set. Taylor diagrams visually compare multiple datasets output to a reference dataset (e.g., model experiments to observations). This is done by exploiting the geometric relationship between centered RMS, correlation, and standard deviation when computing differences between datasets. Several software packages exist to produce Taylor diagrams.</p>
<p>wiki: <a href="https://en.wikipedia.org/wiki/Taylor_diagram" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Taylor_diagram</a></p>
<p>publication: <a href="https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2000JD900719" rel="nofollow noreferrer">https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2000JD900719</a></p>
<p>python, e.g.: <a href="https://pypi.org/project/SkillMetrics/" rel="nofollow noreferrer">https://pypi.org/project/SkillMetrics/</a></p>

CC BY-SA 4.0
2021-03-30T17:44:54.520Z
2021-03-30T17:44:54.520Z
John
<p>You are looking to do some sort of <a href="https://en.wikipedia.org/wiki/Regression_validation" rel="nofollow noreferrer">regression analysis</a> on two-dimensional data. The <span class="math-container">$r^2$</span> <a href="https://en.wikipedia.org/wiki/Coefficient_of_determination" rel="nofollow noreferrer">coefficient of determination</a> value is often used for one dimensional data. <span class="math-container">$r^2=1$</span> means a perfect match and <span class="math-container">$r^2=0$</span> means no correlation. <span class="math-container">$r^2$</span> may be used for two-dimensional data as well; I don't know. Another example is the <a href="https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test" rel="nofollow noreferrer">Kolmogorov–Smirnov test</a>.</p>
<p>Neither of these are, unfortunately, something I know much about but hopefully this will point you in the right direction. What I do know is that this sort of analysis is regularly used for just this situation to assess model output.</p>
<p>I found a <a href="https://aip.scitation.org/doi/pdf/10.1063/1.4822753" rel="nofollow noreferrer">1988 scientific paper</a> by William H. Press and Saul A. Teukolsky entitled <em>Kolmogorov-Smirnov Test For Two-Dimensional Data: How to tell whether a set of (x,y) data points are consistent with a particular probability distribution, or with another data set</em></p>
<p>The paper gives a Numerical Method, basically an algorithm, to apply this to your data. It's not in a modern programming language but you should be able to write the equivalent based on what is described. The <code>R</code> programming language may even come with an inbuilt solution.</p>
<p>The answer in that paper should do what you want but whether it is the <em>best</em> way of doing it, I have no idea.</p>

CC BY-SA 4.0
2021-03-28T11:55:28.137Z
2021-03-30T17:49:26.357Z
2021-03-30T17:49:26.357Z
Mehmet Karatay
<p>I have a number of arrays describing both forecasts and observations of geospatial data (e.g. precipitation intensity, cloud fields...). Included are four animated gifs, each looping back and forth between an observation (of precipitation) &amp; a forecast covering the period of the observation. The four forecasts vary markedly in quality (from extremely good in the first case to terrible in the fourth).</p>
<p>I would like to have some way of systematically quantifying the similarity between two such arrays (e.g. between an observation and its associated forecast). I don't really know any more sophisticated way than to simply take the difference between two arrays (normalized somehow, e.g. by dividing by the mean of one of the first).</p>
<p>The problem is that 'scoring' the forecasts in this way gives case #3 &amp; #4 equally bad marks - even though from visual inspection case #3 appears substantially better than #4. Clouds which are in slightly the wrong place (e.g. the ones in central Germany, or in Switzerland &amp; Western Austria, in case #3) score equally poorly to those which are in <em>entirely</em> the wrong place, or simply absent.</p>
<p>Ideally, I'd like some measure of the minimum amount by which one image needs to be 'deformed' to yield another one, but I have no idea how to approach that in anything like a systematic manner.</p>
<p><a href="https://i.sstatic.net/CHRwP.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/CHRwP.gif" alt="An animated gif comparing observed precipitation to a forecast covering the same period" /></a>
<a href="https://i.sstatic.net/WFQzZ.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/WFQzZ.gif" alt="n animated gif comparing observed precipitation to a forecast covering the same period" /></a>
<a href="https://i.sstatic.net/IYl5d.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/IYl5d.gif" alt="n animated gif comparing observed precipitation to a forecast covering the same period" /></a>
<a href="https://i.sstatic.net/ImPx6.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ImPx6.gif" alt="n animated gif comparing observed precipitation to a forecast covering the same period" /></a></p>

<data-analysis><weatherdata>
Comparing 2D fields - how do you quantify similarity?
Drubbels
# Comparing 2D fields - how do you quantify similarity?

(Asked by: username_0 on 2021-03-27)

I have a number of arrays describing both forecasts and observations of geospatial data (e.g. precipitation intensity, cloud fields...). Included are four animated gifs, each looping back and forth between an observation (of precipitation) & a forecast covering the period of the observation. The four forecasts vary markedly in quality (from extremely good in the first case to terrible in the fourth).

I would like to have some way of systematically quantifying the similarity between two such arrays (e.g. between an observation and its associated forecast). I don't really know any more sophisticated way than to simply take the difference between two arrays (normalized somehow, e.g. by dividing by the mean of one of the first).

The problem is that 'scoring' the forecasts in this way gives case #3 & #4 equally bad marks - even though from visual inspection case #3 appears substantially better than #4. Clouds which are in slightly the wrong place (e.g. the ones in central Germany, or in Switzerland & Western Austria, in case #3) score equally poorly to those which are in *entirely* the wrong place, or simply absent.

Ideally, I'd like some measure of the minimum amount by which one image needs to be 'deformed' to yield another one, but I have no idea how to approach that in anything like a systematic manner.

[![An animated gif comparing observed precipitation to a forecast covering the same period](https://i.sstatic.net/CHRwP.gif)](https://i.sstatic.net/CHRwP.gif)
[![n animated gif comparing observed precipitation to a forecast covering the same period](https://i.sstatic.net/WFQzZ.gif)](https://i.sstatic.net/WFQzZ.gif)
[![n animated gif comparing observed precipitation to a forecast covering the same period](https://i.sstatic.net/IYl5d.gif)](https://i.sstatic.net/IYl5d.gif)
[![n animated gif comparing observed precipitation to a forecast covering the same period](https://i.sstatic.net/ImPx6.gif)](https://i.sstatic.net/ImPx6.gif)

**Tags:** data-analysis, weatherdata


---

## 2 answers

### Answer by username_1 on 2021-03-30. Score: 2

Model data seta can vary in several ways from a reference data set. Taylor diagrams visually compare multiple datasets output to a reference dataset (e.g., model experiments to observations). This is done by exploiting the geometric relationship between centered RMS, correlation, and standard deviation when computing differences between datasets. Several software packages exist to produce Taylor diagrams.

wiki: <https://en.wikipedia.org/wiki/Taylor_diagram>

publication: <https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2000JD900719>

python, e.g.: <https://pypi.org/project/SkillMetrics/>


---

### Answer by username_2 on 2021-03-28. Score: 0

You are looking to do some sort of [regression analysis](https://en.wikipedia.org/wiki/Regression_validation) on two-dimensional data. The $r^2$ [coefficient of determination](https://en.wikipedia.org/wiki/Coefficient_of_determination) value is often used for one dimensional data. $r^2=1$ means a perfect match and $r^2=0$ means no correlation. $r^2$ may be used for two-dimensional data as well; I don't know. Another example is the [Kolmogorov–Smirnov test](https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test).

Neither of these are, unfortunately, something I know much about but hopefully this will point you in the right direction. What I do know is that this sort of analysis is regularly used for just this situation to assess model output.

I found a [1988 scientific paper](https://aip.scitation.org/doi/pdf/10.1063/1.4822753) by William H. Press and Saul A. Teukolsky entitled *Kolmogorov-Smirnov Test For Two-Dimensional Data: How to tell whether a set of (x,y) data points are consistent with a particular probability distribution, or with another data set*

The paper gives a Numerical Method, basically an algorithm, to apply this to your data. It's not in a modern programming language but you should be able to write the equivalent based on what is described. The `R` programming language may even come with an inbuilt solution.

The answer in that paper should do what you want but whether it is the *best* way of doing it, I have no idea.


---

