  This chapter presents an introduction to Markovian modeling for the analysis
of sequence data. Contrary to the deterministic approach seen in the previous
sequence analysis chapters, Markovian models are probabilistic models, focusing
on the transitions between states instead of studying sequences as a whole. The
chapter provides an introduction to this method and differentiates between its
most common variations: first-order Markov models, hidden Markov models,
mixture Markov models, and mixture hidden Markov models. In addition to a
thorough explanation and contextualization within the existing literature, the
chapter provides a step-by-step tutorial on how to implement each type of
Markovian model using the R package seqHMM. The chaper also provides a complete
guide to performing stochastic process mining with Markovian models as well as
plotting, comparing and clustering different process models.
