  We present the Supermasks in Superposition (SupSup) model, capable of
sequentially learning thousands of tasks without catastrophic forgetting. Our
approach uses a randomly initialized, fixed base network and for each task
finds a subnetwork (supermask) that achieves good performance. If task identity
is given at test time, the correct subnetwork can be retrieved with minimal
memory usage. If not provided, SupSup can infer the task using gradient-based
optimization to find a linear superposition of learned supermasks which
minimizes the output entropy. In practice we find that a single gradient step
is often sufficient to identify the correct mask, even among 2500 tasks. We
also showcase two promising extensions. First, SupSup models can be trained
entirely without task identity information, as they may detect when they are
uncertain about new data and allocate an additional supermask for the new
training distribution. Finally the entire, growing set of supermasks can be
stored in a constant-sized reservoir by implicitly storing them as attractors
in a fixed-sized Hopfield network.
