<p>If reanalysis data (<a href="http://www.ecmwf.int/en/research/climate-reanalysis/era-interim" rel="nofollow noreferrer">ERA</a>, <a href="http://data-portal.ecmwf.int/" rel="nofollow noreferrer">ECMWF</a>, <a href="http://cfs.ncep.noaa.gov/cfsr/" rel="nofollow noreferrer">CSFRR</a>) is too aggregate for you, try looking for the datasets used to calibrate photovoltaics models (<a href="http://re.jrc.ec.europa.eu/pvgis/" rel="nofollow noreferrer">PVGIS</a>, <a href="http://pvwatts.nrel.gov/" rel="nofollow noreferrer">PVWATTS</a> and the like) - they include both direct and diffuse insolation.</p>

<p>Also, follow the citation trail for papers that describe the calibration of the PV models, and the reanalysis models - some of the datasets you are seeking will be somewhere in that trail.</p>

<p>Barry Carter has pointed you at the NOAA ISD-Lite dataset, and I'd like to extend that pointer to the whole <a href="https://www.ncdc.noaa.gov/isd" rel="nofollow noreferrer">Integrated Surface Data from NOAA</a>, which is enormous: 20,000 global stations. Because it's an amalgamation of many different sources, they don't all share the same metadata, but <a href="http://www1.ncdc.noaa.gov/pub/data/ish/ish-format-document.pdf" rel="nofollow noreferrer">this pdf</a> lists the breadth of the metadata: the insolation variables are on pages 64-77.</p>

CC BY-SA 3.0
2014-10-23T16:56:32.667Z
2016-11-21T20:25:03.943Z
2016-11-21T20:25:03.943Z
410 gone
<p>It will take effort and programming knowledge to get data from Chilbolton Observatory, Hampshire, England, one of the few cloud research sites in the world. <a href="http://www.stfc.ac.uk/chilbolton/default.aspx" rel="nofollow">http://www.stfc.ac.uk/chilbolton/default.aspx</a></p>

<p>To get data you need to create an account at BADC and stick to the few terms and conditions. <a href="https://badc.nerc.ac.uk/data/" rel="nofollow">https://badc.nerc.ac.uk/data/</a></p>

<p>Data time resolution is logger resolution, sub 1 minute. You will have to decimate. Data is supplied as NETCDF. You get raw data, assume nothing. You will have to detect and handle any missing data. </p>

<p>I think you also need to understand the instruments. (in this case you will need to know and ignore some things which are widely stated and wrong)</p>

<p>Full up and down from direct, indirect and "nocturnal". 
Wind data, yes.
Temperature and simple humidity, yes.
Exotics, yes. </p>

<p>I think reality is going to strike, perhaps you need to consider what you are really trying to do. If you put in the effort I think you will reach few definite conclusions above what you can learn from a lot of reading. The gain is a working understanding of reality.</p>

CC BY-SA 3.0
2014-10-31T02:29:47.210Z
2014-10-31T02:29:47.210Z
TimC
<p>I really need some reliable measured hourly weather data for a few sites across the world. It needs to include solar diffuse and direct values or their equivalents.</p>

<p>I've tried the usual sources: Met Office, Weather Underground and Weather Analytics but none of them provide what I need. They are either not hourly, don't contain solar radiation, or have difficult licence/ access restrictions. This is for some research to determine if there's any relation between position of sun, temperature, relative humidity, wind speed and direction. So I'd need a couple of years data for a couple of locations. Aggregate data isn't much use I'm afraid.</p>

CC BY-SA 3.0
2014-10-23T15:20:37.373Z
Have you looked at the data available at https://earthdata.nasa.gov/ ? Also you may wish to look at specific humidity or water vapor mixing ratios, as relative humidity is dependent on moisture and temperature.
casey
<meteorology>
Obtaining hourly weather data, including direct and diffuse insolation
James
# Obtaining hourly weather data, including direct and diffuse insolation

(Asked by: username_0 on 2014-10-23)

I really need some reliable measured hourly weather data for a few sites across the world. It needs to include solar diffuse and direct values or their equivalents.

I've tried the usual sources: Met Office, Weather Underground and Weather Analytics but none of them provide what I need. They are either not hourly, don't contain solar radiation, or have difficult licence/ access restrictions. This is for some research to determine if there's any relation between position of sun, temperature, relative humidity, wind speed and direction. So I'd need a couple of years data for a couple of locations. Aggregate data isn't much use I'm afraid.

**Tags:** meteorology

> **Comments:**
> **username_1**: Have you looked at the data available at https://earthdata.nasa.gov/ ? Also you may wish to look at specific humidity or water vapor mixing ratios, as relative humidity is dependent on moisture and temperature.  

---

## 2 answers

### Answer by username_2 on 2014-10-23. Score: 8

If reanalysis data ([ERA](http://www.ecmwf.int/en/research/climate-reanalysis/era-interim), [ECMWF](http://data-portal.ecmwf.int/), [CSFRR](http://cfs.ncep.noaa.gov/cfsr/)) is too aggregate for you, try looking for the datasets used to calibrate photovoltaics models ([PVGIS](http://re.jrc.ec.europa.eu/pvgis/), [PVWATTS](http://pvwatts.nrel.gov/) and the like) - they include both direct and diffuse insolation.

Also, follow the citation trail for papers that describe the calibration of the PV models, and the reanalysis models - some of the datasets you are seeking will be somewhere in that trail.

Barry Carter has pointed you at the NOAA ISD-Lite dataset, and I'd like to extend that pointer to the whole [Integrated Surface Data from NOAA](https://www.ncdc.noaa.gov/isd), which is enormous: 20,000 global stations. Because it's an amalgamation of many different sources, they don't all share the same metadata, but [this pdf](http://www1.ncdc.noaa.gov/pub/data/ish/ish-format-document.pdf) lists the breadth of the metadata: the insolation variables are on pages 64-77.


---

### Answer by username_3 on 2014-10-31. Score: 4

It will take effort and programming knowledge to get data from Chilbolton Observatory, Hampshire, England, one of the few cloud research sites in the world. <http://www.stfc.ac.uk/chilbolton/default.aspx>

To get data you need to create an account at BADC and stick to the few terms and conditions. <https://badc.nerc.ac.uk/data/>

Data time resolution is logger resolution, sub 1 minute. You will have to decimate. Data is supplied as NETCDF. You get raw data, assume nothing. You will have to detect and handle any missing data.

I think you also need to understand the instruments. (in this case you will need to know and ignore some things which are widely stated and wrong)

Full up and down from direct, indirect and "nocturnal".
Wind data, yes.
Temperature and simple humidity, yes.
Exotics, yes.

I think reality is going to strike, perhaps you need to consider what you are really trying to do. If you put in the effort I think you will reach few definite conclusions above what you can learn from a lot of reading. The gain is a working understanding of reality.


---

