  We present a framework for the unsupervised learning of neurosymbolic
encoders, which are encoders obtained by composing neural networks with
symbolic programs from a domain-specific language. Our framework naturally
incorporates symbolic expert knowledge into the learning process, which leads
to more interpretable and factorized latent representations compared to fully
neural encoders. We integrate modern program synthesis techniques with the
variational autoencoding (VAE) framework, in order to learn a neurosymbolic
encoder in conjunction with a standard decoder. The programmatic descriptions
from our encoders can benefit many analysis workflows, such as in behavior
modeling where interpreting agent actions and movements is important. We
evaluate our method on learning latent representations for real-world
trajectory data from animal biology and sports analytics. We show that our
approach offers significantly better separation of meaningful categories than
standard VAEs and leads to practical gains on downstream analysis tasks, such
as for behavior classification.
