  Reasoning over an instance composed of a set of vectors, like a point cloud,
requires that one accounts for intra-set dependent features among elements.
However, since such instances are unordered, the elements' features should
remain unchanged when the input's order is permuted. This property, permutation
equivariance, is a challenging constraint for most neural architectures. While
recent work has proposed global pooling and attention-based solutions, these
may be limited in the way that intradependencies are captured in practice. In
this work we propose a more general formulation to achieve permutation
equivariance through ordinary differential equations (ODE). Our proposed
module, Exchangeable Neural ODE (ExNODE), can be seamlessly applied for both
discriminative and generative tasks. We also extend set modeling in the
temporal dimension and propose a VAE based model for temporal set modeling.
Extensive experiments demonstrate the efficacy of our method over strong
baselines.
