<p>I'm part of an organization maintaning a large network of weather stations (~3000 stations) over a European country.
Besides storing data measured from the stations into a database, we also produce maps of interpolated values over a 2.5 km regular grid. At the moment we're using the Natural Neighbour algorithm from pyngl (<code>natgrid</code>, <a href="https://www.pyngl.ucar.edu/Functions/Ngl.natgrid.shtml" rel="nofollow noreferrer">https://www.pyngl.ucar.edu/Functions/Ngl.natgrid.shtml</a>).</p>
<p>While most of the variables are directly interpolated on the grid, the temperature is always scaled with the height in order to obtain a better representation of the gradient over mountainous regions (see an example image)</p>
<p><a href="https://i.sstatic.net/XnkvF.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/XnkvF.png" alt="enter image description here" /></a></p>
<p>I've been trying to find a better way to interpolate the data without success. My ideal algorithm should be able to</p>
<ul>
<li>take scattered data (as temperature, rain, humidity...) as input: data is varying between 1000 and 5000 points depending on the time of the day</li>
<li>add external parameters such as altitude or latitude to improve the interpolation of data over area where station data is missing (like mountainous regions), similar to what done with kriging drifts. Ideally we could even use radar data to correct precipitation estimates. This should be tuned automatically by the algorithm itself without the need to intervene</li>
<li>be fast (less than 10 seconds to interpolate all data)</li>
<li>ideally use a machine learning model trained on the entire archive of the data to improve the interpolation? (this is optional)</li>
</ul>
<p>I've been trying with RBF interpolation, all the methods from <code>metpy</code> (<a href="https://unidata.github.io/MetPy/latest/examples/gridding/Point_Interpolation.html" rel="nofollow noreferrer">https://unidata.github.io/MetPy/latest/examples/gridding/Point_Interpolation.html</a>), and even <code>pykrige</code> with Ordinary Kriging or Regression Kriging with a Random Forest regressor. Somehow I could not find a method that satisfies all the previous conditions: <code>natgrid</code> still seems to be the fastest and most accurate algorithm.</p>
<p>Can someone point me in the right direction?</p>

CC BY-SA 4.0
2022-06-28T08:22:13.453Z
Hi did you tried Universal Kriging 3D? Also included in the pykrige package
Weiss
CC BY-SA 4.0
2022-06-29T08:18:48.247Z
Yes, I did. But somehow because of either the number of stations or grid resolution it was super slow (like 5 mins)
Droid
CC BY-SA 4.0
2022-06-29T08:33:09.097Z
write something like: def generate_grid(data, basemap, delta=1):
    grid = {
        'lon': np.arange(-30, 60, delta),
        'lat': np.arange(30,70, delta) # dont extrapolate towards the poles
    }
    grid["x"], grid["y"] = np.meshgrid(grid["lon"], grid["lat"])
    grid["x"], grid["y"] = basemap(grid["x"], grid["y"]) with     grid = generate_grid(base_data, basemap,0.1)
Weiss
CC BY-SA 4.0
2022-06-29T09:35:02.140Z
sorry, I'm not looking to define a new grid but rather to find a method that works efficiently on any 
Droid
CC BY-SA 4.0
2022-06-29T09:50:25.557Z
If I were you, I would work with what you have! And to calculate the altitude, I would suggest you find a formula that describes the temperature change as a function of altitude. Then I would calculate all the station data down to sea level, interpolate them and at the end put altitude data under the interpolated data and calculate them up again with the same altitude formula.
Weiss
CC BY-SA 4.0
2022-06-29T12:06:41.690Z
@Weiss, sorry, maybe I wasn't clear enough in my description. This is exactly what I'm doing right now, but with an external drift it could be done automatically.
Droid
CC BY-SA 4.0
2023-09-01T01:36:41.387Z
I realize digging up an old thread here, but this is closely coupled to a problem I’m trying to solve as well. Did you ever find a solution? The US has some sensor dense regions where interpolation methods may be more representative of tertiary effects. From which a model could be trained with with the topo map layer then applied to data sparse regions.
Jason Leaver
CC BY-SA 4.0
2023-09-01T14:25:16.897Z
I've skimmed over the whole internet..While there are the capabilities to do something like this I haven't been able to find a method with the requirements listed above..At the moment we're still using something hybrid based on Rbf interpolation and some adjustment with orography. Everything else was just too slow or too complicated. I've been experimenting with a new method using a weighted linear regressions of temperature with height (using weights like distance, elevation, sea distance...).and it works better in some situations but it's just too slow.
Droid
<meteorology><temperature><precipitation><interpolation>
Best method for weather station data interpolation with external parameters
Droid
# Best method for weather station data interpolation with external parameters

(Asked by: username_0 on 2022-06-27)

I'm part of an organization maintaning a large network of weather stations (~3000 stations) over a European country.
Besides storing data measured from the stations into a database, we also produce maps of interpolated values over a 2.5 km regular grid. At the moment we're using the Natural Neighbour algorithm from pyngl (`natgrid`, <https://www.pyngl.ucar.edu/Functions/Ngl.natgrid.shtml>).

While most of the variables are directly interpolated on the grid, the temperature is always scaled with the height in order to obtain a better representation of the gradient over mountainous regions (see an example image)

[![enter image description here](https://i.sstatic.net/XnkvF.png)](https://i.sstatic.net/XnkvF.png)

I've been trying to find a better way to interpolate the data without success. My ideal algorithm should be able to

* take scattered data (as temperature, rain, humidity...) as input: data is varying between 1000 and 5000 points depending on the time of the day
* add external parameters such as altitude or latitude to improve the interpolation of data over area where station data is missing (like mountainous regions), similar to what done with kriging drifts. Ideally we could even use radar data to correct precipitation estimates. This should be tuned automatically by the algorithm itself without the need to intervene
* be fast (less than 10 seconds to interpolate all data)
* ideally use a machine learning model trained on the entire archive of the data to improve the interpolation? (this is optional)

I've been trying with RBF interpolation, all the methods from `metpy` (<https://unidata.github.io/MetPy/latest/examples/gridding/Point_Interpolation.html>), and even `pykrige` with Ordinary Kriging or Regression Kriging with a Random Forest regressor. Somehow I could not find a method that satisfies all the previous conditions: `natgrid` still seems to be the fastest and most accurate algorithm.

Can someone point me in the right direction?

**Tags:** meteorology, temperature, precipitation, interpolation

> **Comments:**
> **username_1**: Hi did you tried Universal Kriging 3D? Also included in the pykrige package  
> **username_0**: Yes, I did. But somehow because of either the number of stations or grid resolution it was super slow (like 5 mins)  
> **username_1**: write something like: def generate_grid(data, basemap, delta=1):     grid = {         'lon': np.arange(-30, 60, delta),         'lat': np.arange(30,70, delta) # dont extrapolate towards the poles     }     grid["x"], grid["y"] = np.meshgrid(grid["lon"], grid["lat"])     grid["x"], grid["y"] = basemap(grid["x"], grid["y"]) with     grid = generate_grid(base_data, basemap,0.1)  
> **username_0**: sorry, I'm not looking to define a new grid but rather to find a method that works efficiently on any   
> **username_1**: If I were you, I would work with what you have! And to calculate the altitude, I would suggest you find a formula that describes the temperature change as a function of altitude. Then I would calculate all the station data down to sea level, interpolate them and at the end put altitude data under the interpolated data and calculate them up again with the same altitude formula.  
> **username_0**: @username_1, sorry, maybe I wasn't clear enough in my description. This is exactly what I'm doing right now, but with an external drift it could be done automatically.  
> **username_2**: I realize digging up an old thread here, but this is closely coupled to a problem I’m trying to solve as well. Did you ever find a solution? The US has some sensor dense regions where interpolation methods may be more representative of tertiary effects. From which a model could be trained with with the topo map layer then applied to data sparse regions.  
> **username_0**: I've skimmed over the whole internet..While there are the capabilities to do something like this I haven't been able to find a method with the requirements listed above..At the moment we're still using something hybrid based on Rbf interpolation and some adjustment with orography. Everything else was just too slow or too complicated. I've been experimenting with a new method using a weighted linear regressions of temperature with height (using weights like distance, elevation, sea distance...).and it works better in some situations but it's just too slow.  

---

## 0 answers

