  Stability of recurrent models is closely linked with trainability,
generalizability and in some applications, safety. Methods that train stable
recurrent neural networks, however, do so at a significant cost to
expressibility. We propose an implicit model structure that allows for a convex
parametrization of stable models using contraction analysis of non-linear
systems. Using these stability conditions we propose a new approach to model
initialization and then provide a number of empirical results comparing the
performance of our proposed model set to previous stable RNNs and vanilla RNNs.
By carefully controlling stability in the model, we observe a significant
increase in the speed of training and model performance.
