  Time-lagged autoencoders (TAEs) have been proposed as a deep learning
regression-based approach to the discovery of slow modes in dynamical systems.
However, a rigorous analysis of nonlinear TAEs remains lacking. In this work,
we discuss the capabilities and limitations of TAEs through both theoretical
and numerical analyses. Theoretically, we derive bounds for nonlinear TAE
performance in slow mode discovery and show that in general TAEs learn a
mixture of slow and maximum variance modes. Numerically, we illustrate cases
where TAEs can and cannot correctly identify the leading slowest mode in two
example systems: a 2D "Washington beltway" potential and the alanine dipeptide
molecule in explicit water. We also compare the TAE results with those obtained
using state-free reversible VAMPnets (SRVs) as a variational-based neural
network approach for slow modes discovery, and show that SRVs can correctly
discover slow modes where TAEs fail.
