  We survey current developments in the approximation theory of sequence
modelling in machine learning. Particular emphasis is placed on classifying
existing results for various model architectures through the lens of classical
approximation paradigms, and the insights one can gain from these results. We
also outline some future research directions towards building a theory of
sequence modelling.
