  We present a method for storing multiple models within a single set of
parameters. Models can coexist in superposition and still be retrieved
individually. In experiments with neural networks, we show that a surprisingly
large number of models can be effectively stored within a single parameter
instance. Furthermore, each of these models can undergo thousands of training
steps without significantly interfering with other models within the
superposition. This approach may be viewed as the online complement of
compression: rather than reducing the size of a network after training, we make
use of the unrealized capacity of a network during training.
