  The meaning of a sentence is a function of the relations that hold between
its words. We instantiate this relational view of semantics in a series of
neural models based on variants of relation networks (RNs) which represent a
set of objects (for us, words forming a sentence) in terms of representations
of pairs of objects. We propose two extensions to the basic RN model for
natural language. First, building on the intuition that not all word pairs are
equally informative about the meaning of a sentence, we use constraints based
on both supervised and unsupervised dependency syntax to control which
relations influence the representation. Second, since higher-order relations
are poorly captured by a sum of pairwise relations, we use a recurrent
extension of RNs to propagate information so as to form representations of
higher order relations. Experiments on sentence classification, sentence pair
classification, and machine translation reveal that, while basic RNs are only
modestly effective for sentence representation, recurrent RNs with latent
syntax are a reliably powerful representational device.
