  Past work probing compositionality in sentence embedding models faces issues
determining the causal impact of implicit syntax representations. Given a
sentence, we construct a neural module net based on its syntax parse and train
it end-to-end to approximate the sentence's embedding generated by a
transformer model. The distillability of a transformer to a Syntactic NeurAl
Module Net (SynNaMoN) then captures whether syntax is a strong causal model of
its compositional ability. Furthermore, we address questions about the geometry
of semantic composition by specifying individual SynNaMoN modules' internal
architecture & linearity. We find differences in the distillability of various
sentence embedding models that broadly correlate with their performance, but
observe that distillability doesn't considerably vary by model size. We also
present preliminary evidence that much syntax-guided composition in sentence
embedding models is linear, and that non-linearities may serve primarily to
handle non-compositional phrases.
