  Kriging and Gaussian Process Regression are statistical methods that allow
predicting the outcome of a random process or a random field by using a sample
of correlated observations. In other words, the random process or random field
is partially observed, and by using a sample a prediction is made, pointwise or
as a whole, where the latter can be thought as a reconstruction. In addition,
the techniques permit to give a measure of uncertainty of the prediction. The
methods have different origins. Kriging comes from geostatistics, a field which
started to develop around 1950 oriented to mining valuation problems, whereas
Gaussian Process Regression has gained popularity in the area of machine
learning in the last decade of the previous century. In the literature, the
methods are usually presented as being the same technique. However, beyond this
affirmation, the techniques have yet not been compared on a thorough
mathematical basis and neither explained why and under which conditions this
affirmation holds. Furthermore, Kriging has many variants and this affirmation
should be precised. In this paper, this gap is filled. It is shown, step by
step how both methods are deduced from the first principles -- with a major
focus on Kriging, the mathematical connection between them, and which Kriging
variant corresponds to which Gaussian Process Regression set up. The three most
widely used versions of Kriging are considered: Simple Kriging, Ordinary
Kriging and Universal Kriging. It is found, that despite their closeness, the
techniques are different in their approach and assumptions, in a similar way
the Least Square method, the Best Linear Unbiased Estimator method, and the
Likelihood method in regression do. I hope this work can serve for a deeper
understanding of the relationship between Kriging and Gaussian Process
Regression, as well as a cohesive introductory resource for researchers.
