  This study shows that a mixture of RNN experts model can acquire the ability
to generate sequences combining multiple primitive patterns by means of
self-organizing chaos. By training of the model, each expert learns a primitive
sequence pattern, and a gating network learns to imitate stochastic switching
of the multiple primitives via a chaotic dynamics, utilizing a sensitive
dependence on initial conditions. As a demonstration, we present a numerical
simulation in which the model learns Markov chain switching among some
Lissajous curves by a chaotic dynamics. Our analysis shows that by using a
sufficient amount of training data, balanced with the network memory capacity,
it is possible to satisfy the conditions for embedding the target stochastic
sequences into a chaotic dynamical system. It is also shown that reconstruction
of a stochastic time series by a chaotic model can be stabilized by adding a
negligible amount of noise to the dynamics of the model.
