  Scientific documents rely on both mathematics and text to communicate ideas.
Inspired by the topical correspondence between mathematical equations and word
contexts observed in scientific texts, we propose a novel topic model that
jointly generates mathematical equations and their surrounding text (TopicEq).
Using an extension of the correlated topic model, the context is generated from
a mixture of latent topics, and the equation is generated by an RNN that
depends on the latent topic activations. To experiment with this model, we
create a corpus of 400K equation-context pairs extracted from a range of
scientific articles from arXiv, and fit the model using a variational
autoencoder approach. Experimental results show that this joint model
significantly outperforms existing topic models and equation models for
scientific texts. Moreover, we qualitatively show that the model effectively
captures the relationship between topics and mathematics, enabling novel
applications such as topic-aware equation generation, equation topic inference,
and topic-aware alignment of mathematical symbols and words.
