  The interpolation of spatial data can be of tremendous value in various
applications, such as forecasting weather from only a few measurements of
meteorological or remote sensing data. Existing methods for spatial
interpolation, such as variants of kriging and spatial autoregressive models,
tend to suffer from at least one of the following limitations: (a) the
assumption of stationarity, (b) the assumption of isotropy, and (c) the
trade-off between modelling local or global spatial interaction. Addressing
these issues in this work, we propose the use of Markov reward processes (MRPs)
as a spatial interpolation method, and we introduce three variants thereof: (i)
a basic static discount MRP (SD-MRP), (ii) an accurate but mostly theoretical
optimised MRP (O-MRP), and (iii) a transferable weight prediction MRP (WP-MRP).
All variants of MRP interpolation operate locally, while also implicitly
accounting for global spatial relationships in the entire system through
recursion. Additionally, O-MRP and WP-MRP no longer assume stationarity and are
robust to anisotropy. We evaluated our proposed methods by comparing the mean
absolute errors of their interpolated grid cells to those of 7 common
baselines, selected from models based on spatial autocorrelation, (spatial)
regression, and deep learning.
  We performed detailed evaluations on two publicly available datasets (local
GDP values, and COVID-19 patient trajectory data). The results from these
experiments clearly show the competitive advantage of MRP interpolation, which
achieved significantly lower errors than the existing methods in 23 out of 40
experimental conditions, or 35 out of 40 when including O-MRP.
