  For applications in chemistry and physics, machine learning (ML) is generally
used to solve one of three problems: interpolation, classification or
clustering. These problems use information about physical systems in a certain
range of parameters or variables in order to make predictions at unknown values
of these variables within the same range. The present work illustrates the
application of ML to prediction of physical properties outside the range of the
training parameters. We define `physical extrapolation' to refer to accurate
predictions $y(\mathbf{x^\ast})$ of a given physical property at a point
$\mathbf{x^\ast} = [ x^\ast_1, ..., x^\ast_{\cal D} ]$ in the $\cal
D$-dimensional space, if, at least, one of the variables $x^\ast_i \in \left [
x^\ast_1, ..., x^\ast_{\cal D} \right ]$ is {\it outside} of the range covering
the training data. We show that Gaussian processes (GPs) can be used to build
ML models capable of physical extrapolation of quantum properties of complex
systems across quantum phase transitions. The approach is based on training GP
models of variable complexity by the evolution of the physical functions. We
show that, as the complexity of the models increases, they become capable of
predicting new transitions. We also show that, where the evolution of the
physical functions is analytic and relatively simple, GP models with simple
kernels already yield accurate generalization results, allowing for accurate
predictions of quantum properties in a different quantum phase. For more
complex problems, it is necessary to build models with complex kernels. The
complexity of the kernels is increased using the Bayesian Information Criterion
(BIC). We illustrate the importance of the BIC by comparing the results with
random kernels of various complexity and illustrate a method to obtain
meaningful extrapolation results without direct validation in the extrapolated
region.
