<p>Unfortunately, while there's lots of coordination of the CMIP6 experiment protocols across the modelling groups, there's little coordination of the evaluation of outputs, which is done in different ways by different groups.  There's a
<a href="https://agupubs.onlinelibrary.wiley.com/topic/vi-categories-19422466/earth-system-modeling-2018-2020/19422466" rel="nofollow noreferrer">JAMES Special Collection</a> of Special Issues covering some models (CAS-ESM, CESM2, IPSL, GISS, CNRM, E3SM, GFDL, HadGEM3, MPI-ESM1.2) and several GMD Special Issues for individual models (e.g., <a href="https://gmd.copernicus.org/articles/special_issue989.html" rel="nofollow noreferrer">CanESM</a>, <a href="https://gmd.copernicus.org/articles/special_issue20.html" rel="nofollow noreferrer">NorESM</a>, <a href="https://gmd.copernicus.org/articles/special_issue1027.html" rel="nofollow noreferrer">ACCESS</a>).  These are used differently by each group; some use them just to document the model, some also use them for model evaluation and science results.</p>
<p>EC-Earth seems to be notable for an absence of a special issue.  There's a paper on <a href="https://gmd.copernicus.org/preprints/gmd-2020-446/" rel="nofollow noreferrer">EC-Earth3 historical simulations</a> and an <a href="https://eartharxiv.org/repository/view/267/" rel="nofollow noreferrer">EC-Earth3 report</a> but nothing obviously coordinated.  The information is probably scattered across various papers and journals.</p>
<p>It's worth noting that CMIP6 outputs are pushed through various standard benchmarking tools and put on the web, which you might find useful: e.g., <a href="https://www.ilamb.org/CMIP6/historical" rel="nofollow noreferrer">https://www.ilamb.org/CMIP6/historical</a>.</p>
<blockquote>
<p>Is it possible that the data from these models contribute to CMIP6 without any peer-reviewed evaluation of the outputs?</p>
</blockquote>
<p>Yes and no.  Pretty much every group will have some form of internal evaluation and sanity check of their model outputs before publishing them to the ESGF.  Some groups require a human to approve every single file that gets published.  But with each CMIP6 model producing petabytes of data, this internal post-processing, review and publication work flow requires a significant amount of effort by each group (e.g., <a href="https://opensky.ucar.edu/islandora/object/conference%3A2967/datastream/PDF/view" rel="nofollow noreferrer">CESM at NCAR</a>).  It's just not practical for all that data to also be formally peer-reviewed just for the purpose of sitting in a database.</p>
<p>In practice though, each science paper that uses those data is a form of peer-review (because none of us should use any data without sanity checking it for ourselves), and those papers will themselves have traditional, formal peer-review before being published.  (I've had to contact other modelling groups about oddities in their data I've found while writing a paper and sometimes they have the resources to fix the dataset, sometimes they don't.)  So you could say that these datasets have at least three levels of review, and probably more.  One of the benefits of making these data on the ESGF available to anyone is that they are scrutinized by far more people, friendly and unfriendly, than happens under traditional peer-review.</p>

CC BY-SA 4.0
2021-11-19T13:58:55.257Z
2021-11-19T13:58:55.257Z
Deditos
<p>I want to see how well different CMIP6 model realisations did during the model evaluation, but I can't find papers associated with the data presented on the ESGF node. For example, if I was interested in the EC-Earth3 model and specific variant, where can I find out how it performed?</p>
<p>I haven't found an article specifically referring to EC-Earth3, but rather one referring to the AerChemMIP realisation, as opposed to the ScenarioMIP realisation I am interested in. Is it possible that the data from these models contribute to CMIP6 without any peer-reviewed evaluation of the outputs?</p>

<climate>
Where can I find the journal articles associated with CMIP6 models available on World Climate Reseaerch Programme ESGF website?
matlabcat
# Where can I find the journal articles associated with CMIP6 models available on World Climate Reseaerch Programme ESGF website?

(Asked by: username_0 on 2021-11-17)

I want to see how well different CMIP6 model realisations did during the model evaluation, but I can't find papers associated with the data presented on the ESGF node. For example, if I was interested in the EC-Earth3 model and specific variant, where can I find out how it performed?

I haven't found an article specifically referring to EC-Earth3, but rather one referring to the AerChemMIP realisation, as opposed to the ScenarioMIP realisation I am interested in. Is it possible that the data from these models contribute to CMIP6 without any peer-reviewed evaluation of the outputs?

**Tags:** climate


---

## 1 answers

### Answer by username_1 on 2021-11-19. Score: 2 | Accepted answer

Unfortunately, while there's lots of coordination of the CMIP6 experiment protocols across the modelling groups, there's little coordination of the evaluation of outputs, which is done in different ways by different groups. There's a
[JAMES Special Collection](https://agupubs.onlinelibrary.wiley.com/topic/vi-categories-19422466/earth-system-modeling-2018-2020/19422466) of Special Issues covering some models (CAS-ESM, CESM2, IPSL, GISS, CNRM, E3SM, GFDL, HadGEM3, MPI-ESM1.2) and several GMD Special Issues for individual models (e.g., [CanESM](https://gmd.copernicus.org/articles/special_issue989.html), [NorESM](https://gmd.copernicus.org/articles/special_issue20.html), [ACCESS](https://gmd.copernicus.org/articles/special_issue1027.html)). These are used differently by each group; some use them just to document the model, some also use them for model evaluation and science results.

EC-Earth seems to be notable for an absence of a special issue. There's a paper on [EC-Earth3 historical simulations](https://gmd.copernicus.org/preprints/gmd-2020-446/) and an [EC-Earth3 report](https://eartharxiv.org/repository/view/267/) but nothing obviously coordinated. The information is probably scattered across various papers and journals.

It's worth noting that CMIP6 outputs are pushed through various standard benchmarking tools and put on the web, which you might find useful: e.g., <https://www.ilamb.org/CMIP6/historical>.

> Is it possible that the data from these models contribute to CMIP6 without any peer-reviewed evaluation of the outputs?

Yes and no. Pretty much every group will have some form of internal evaluation and sanity check of their model outputs before publishing them to the ESGF. Some groups require a human to approve every single file that gets published. But with each CMIP6 model producing petabytes of data, this internal post-processing, review and publication work flow requires a significant amount of effort by each group (e.g., [CESM at NCAR](https://opensky.ucar.edu/islandora/object/conference%3A2967/datastream/PDF/view)). It's just not practical for all that data to also be formally peer-reviewed just for the purpose of sitting in a database.

In practice though, each science paper that uses those data is a form of peer-review (because none of us should use any data without sanity checking it for ourselves), and those papers will themselves have traditional, formal peer-review before being published. (I've had to contact other modelling groups about oddities in their data I've found while writing a paper and sometimes they have the resources to fix the dataset, sometimes they don't.) So you could say that these datasets have at least three levels of review, and probably more. One of the benefits of making these data on the ESGF available to anyone is that they are scrutinized by far more people, friendly and unfriendly, than happens under traditional peer-review.


---

