  Implicit models are a general class of learning models that forgo the
hierarchical layer structure typical in neural networks and instead define the
internal states based on an ``equilibrium'' equation, offering competitive
performance and reduced memory consumption. However, training such models
usually relies on expensive implicit differentiation for backward propagation.
In this work, we present a new approach to training implicit models, called
State-driven Implicit Modeling (SIM), where we constrain the internal states
and outputs to match that of a baseline model, circumventing costly backward
computations. The training problem becomes convex by construction and can be
solved in a parallel fashion, thanks to its decomposable structure. We
demonstrate how the SIM approach can be applied to significantly improve
sparsity (parameter reduction) and robustness of baseline models trained on
FashionMNIST and CIFAR-100 datasets.
