  Humans rely on strong inductive biases to learn from few examples and
abstract useful information from sensory data. Instilling such biases in
machine learning models has been shown to improve their performance on various
benchmarks including few-shot learning, robustness, and alignment. However,
finding effective training procedures to achieve that goal can be challenging
as psychologically-rich training data such as human similarity judgments are
expensive to scale, and Bayesian models of human inductive biases are often
intractable for complex, realistic domains. Here, we address this challenge by
introducing a Bayesian notion of generative similarity whereby two datapoints
are considered similar if they are likely to have been sampled from the same
distribution. This measure can be applied to complex generative processes,
including probabilistic programs. We show that generative similarity can be
used to define a contrastive learning objective even when its exact form is
intractable, enabling learning of spatial embeddings that express specific
inductive biases. We demonstrate the utility of our approach by showing how it
can be used to capture human inductive biases for geometric shapes, and to
better distinguish different abstract drawing styles that are parameterized by
probabilistic programs.
