  Convolutional network are the de-facto standard for analysing spatio-temporal
data such as images, videos, 3D shapes, etc. Whilst some of this data is
naturally dense (for instance, photos), many other data sources are inherently
sparse. Examples include pen-strokes forming on a piece of paper, or (colored)
3D point clouds that were obtained using a LiDAR scanner or RGB-D camera.
Standard "dense" implementations of convolutional networks are very inefficient
when applied on such sparse data. We introduce a sparse convolutional operation
tailored to processing sparse data that differs from prior work on sparse
convolutional networks in that it operates strictly on submanifolds, rather
than "dilating" the observation with every layer in the network. Our empirical
analysis of the resulting submanifold sparse convolutional networks shows that
they perform on par with state-of-the-art methods whilst requiring
substantially less computation.
