  Continuous input signals like images and time series that are irregularly
sampled or have missing values are challenging for existing deep learning
methods. Coherently defined feature representations must depend on the values
in unobserved regions of the input. Drawing from the work in probabilistic
numerics, we propose Probabilistic Numeric Convolutional Neural Networks which
represent features as Gaussian processes (GPs), providing a probabilistic
description of discretization error. We then define a convolutional layer as
the evolution of a PDE defined on this GP, followed by a nonlinearity. This
approach also naturally admits steerable equivariant convolutions under e.g.
the rotation group. In experiments we show that our approach yields a $3\times$
reduction of error from the previous state of the art on the SuperPixel-MNIST
dataset and competitive performance on the medical time series dataset
PhysioNet2012.
