  Deep equilibrium (DEQ) models have emerged as a promising class of implicit
layer models, which abandon traditional depth by solving for the fixed points
of a single nonlinear layer. Despite their success, the stability of the fixed
points for these models remains poorly understood. By considering DEQ models as
nonlinear dynamic systems, we propose a robust DEQ model named LyaDEQ with
guaranteed provable stability via Lyapunov theory. The crux of our method is
ensuring the Lyapunov stability of the DEQ model's fixed points, which enables
the proposed model to resist minor initial perturbations. To avoid poor
adversarial defense due to Lyapunov-stable fixed points being located near each
other, we orthogonalize the layers after the Lyapunov stability module to
separate different fixed points. We evaluate LyaDEQ models under well-known
adversarial attacks, and experimental results demonstrate significant
improvement in robustness. Furthermore, we show that the LyaDEQ model can be
combined with other defense methods, such as adversarial training, to achieve
even better adversarial robustness.
