  We consider Markov models of stochastic processes where the next-step
conditional distribution is defined by a kernel density estimator (KDE),
similar to Markov forecast densities and certain time-series bootstrap schemes.
The KDE Markov models (KDE-MMs) we discuss are nonlinear, nonparametric, fully
probabilistic representations of stationary processes, based on techniques with
strong asymptotic consistency properties. The models generate new data by
concatenating points from the training data sequences in a context-sensitive
manner, together with some additive driving noise. We present novel EM-type
maximum-likelihood algorithms for data-driven bandwidth selection in KDE-MMs.
Additionally, we augment the KDE-MMs with a hidden state, yielding a new model
class, KDE-HMMs. The added state variable captures non-Markovian long memory
and signal structure (e.g., slow oscillations), complementing the short-range
dependences described by the Markov process. The resulting joint Markov and
hidden-Markov structure is appealing for modelling complex real-world processes
such as speech signals. We present guaranteed-ascent EM-update equations for
model parameters in the case of Gaussian kernels, as well as relaxed update
formulas that greatly accelerate training in practice. Experiments demonstrate
increased held-out set probability for KDE-HMMs on several challenging natural
and synthetic data series, compared to traditional techniques such as
autoregressive models, HMMs, and their combinations.
