  Non-stationary systems are found throughout the world, from climate patterns
under the influence of variation in carbon dioxide concentration, to brain
dynamics driven by ascending neuromodulation. Accordingly, there is a need for
methods to analyze non-stationary processes, and yet most time-series analysis
methods that are used in practice, on important problems across science and
industry, make the simplifying assumption of stationarity. One important
problem in the analysis of non-stationary systems is the problem class that we
refer to as Parameter Inference from a Non-stationary Unknown Process (PINUP).
Given an observed time series, this involves inferring the parameters that
drive non-stationarity of the time series, without requiring knowledge or
inference of a mathematical model of the underlying system. Here we review and
unify a diverse literature of algorithms for PINUP. We formulate the problem,
and categorize the various algorithmic contributions. This synthesis will allow
researchers to identify gaps in the literature and will enable systematic
comparisons of different methods. We also demonstrate that the most common
systems that existing methods are tested on - notably the non-stationary Lorenz
process and logistic map - are surprisingly easy to perform well on using
simple statistical features like windowed mean and variance, undermining the
practice of using good performance on these systems as evidence of algorithmic
performance. We then identify more challenging problems that many existing
methods perform poorly on and which can be used to drive methodological
advances in the field. Our results unify disjoint scientific contributions to
analyzing non-stationary systems and suggest new directions for progress on the
PINUP problem and the broader study of non-stationary phenomena.
