  The fast adaptation capability of deep neural networks in non-stationary
environments is critical for online time series forecasting. Successful
solutions require handling changes to new and recurring patterns. However,
training deep neural forecaster on the fly is notoriously challenging because
of their limited ability to adapt to non-stationary environments and the
catastrophic forgetting of old knowledge. In this work, inspired by the
Complementary Learning Systems (CLS) theory, we propose Fast and Slow learning
Networks (FSNet), a holistic framework for online time-series forecasting to
simultaneously deal with abrupt changing and repeating patterns. Particularly,
FSNet improves the slowly-learned backbone by dynamically balancing fast
adaptation to recent changes and retrieving similar old knowledge. FSNet
achieves this mechanism via an interaction between two complementary components
of an adapter to monitor each layer's contribution to the lost, and an
associative memory to support remembering, updating, and recalling repeating
events. Extensive experiments on real and synthetic datasets validate FSNet's
efficacy and robustness to both new and recurring patterns. Our code is
available at \url{https://github.com/salesforce/fsnet}.
