  Image convolutions have been a cornerstone of a great number of deep learning
advances in computer vision. The research community is yet to settle on an
equivalent operator for sparse, unstructured continuous data like point clouds
and event streams however. We present an elegant sparse matrix-based
interpretation of the convolution operator for these cases, which is consistent
with the mathematical definition of convolution and efficient during training.
On benchmark point cloud classification problems we demonstrate networks built
with these operations can train an order of magnitude or more faster than top
existing methods, whilst maintaining comparable accuracy and requiring a tiny
fraction of the memory. We also apply our operator to event stream processing,
achieving state-of-the-art results on multiple tasks with streams of hundreds
of thousands of events.
