  We introduce semi-parametric inducing point networks (SPIN), a
general-purpose architecture that can query the training set at inference time
in a compute-efficient manner. Semi-parametric architectures are typically more
compact than parametric models, but their computational complexity is often
quadratic. In contrast, SPIN attains linear complexity via a cross-attention
mechanism between datapoints inspired by inducing point methods. Querying large
training sets can be particularly useful in meta-learning, as it unlocks
additional training signal, but often exceeds the scaling limits of existing
models. We use SPIN as the basis of the Inducing Point Neural Process, a
probabilistic model which supports large contexts in meta-learning and achieves
high accuracy where existing models fail. In our experiments, SPIN reduces
memory requirements, improves accuracy across a range of meta-learning tasks,
and improves state-of-the-art performance on an important practical problem,
genotype imputation.
