  A nonparametric method to predict non-Markovian time series of partially
observed dynamics is developed. The prediction problem we consider is a
supervised learning task of finding a regression function that takes a delay
embedded observable to the observable at a future time. When delay embedding
theory is applicable, the proposed regression function is a consistent
estimator of the flow map induced by the delay embedding. Furthermore, the
corresponding Mori-Zwanzig equation governing the evolution of the observable
simplifies to only a Markovian term, represented by the regression function. We
realize this supervised learning task with a class of kernel-based linear
estimators, the kernel analog forecast (KAF), which are consistent in the limit
of large data. In a scenario with a high-dimensional covariate space, we employ
a Markovian kernel smoothing method which is computationally cheaper than the
Nystr\"om projection method for realizing KAF. In addition to the guaranteed
theoretical convergence, we numerically demonstrate the effectiveness of this
approach on higher-dimensional problems where the relevant kernel features are
difficult to capture with the Nystr\"om method. Given noisy training data, we
propose a nonparametric smoother as a de-noising method. Numerically, we show
that the proposed smoother is more accurate than EnKF and 4Dvar in de-noising
signals corrupted by independent (but not necessarily identically distributed)
noise, even if the smoother is constructed using a data set corrupted by white
noise. We show skillful prediction using the KAF constructed from the denoised
data.
