  Sequential transitions between metastable states are ubiquitously observed in
the neural system and underlie various cognitive functions. Although a number
of studies with asymmetric Hebbian connectivity have investigated how such
sequences are generated, the focused sequences are simple Markov ones. On the
other hand, supervised machine learning methods can generate complex non-Markov
sequences, but these sequences are vulnerable against perturbations. Further,
concatenation of newly learned sequence to the already learned one is difficult
due to catastrophe forgetting, although concatenation is essential for
cognitive functions such as inference. How stable complex sequences are
generated still remains unclear. We have developed a neural network with fast
and slow dynamics, which are inspired by the experiments. The slow dynamics
store history of inputs and outputs and affect the fast dynamics depending on
the stored history. We show the learning rule that requires only local
information can form the network generating the complex and robust sequences in
the fast dynamics. The slow dynamics work as bifurcation parameters for the
fast one, wherein they stabilize the next pattern of the sequence before the
current pattern is destabilized. This co-existence period leads to the stable
transition between the current and the next pattern in the sequence. We further
find that timescale balance is critical to this period. Our study provides a
novel mechanism generating the robust complex sequences with multiple
timescales in neural dynamics. Considering the multiple timescales are widely
observed, the mechanism advances our understanding of temporal processing in
the neural system.
