  The analogy between Gaussian processes (GPs) and deep artificial neural
networks (ANNs) has received a lot of interest, and has shown promise to unbox
the blackbox of deep ANNs. Existing theoretical works put strict assumptions on
the ANN (e.g. requiring all intermediate layers to be wide, or using specific
activation functions). Accommodating those theoretical assumptions is hard in
recent deep architectures, and those theoretical conditions need refinement as
new deep architectures emerge. In this paper we derive an evidence lower-bound
that encourages the GP's posterior to match the ANN's output without any
requirement on the ANN. Using our method we find out that on 5 datasets, only a
subset of those theoretical assumptions are sufficient. Indeed, in our
experiments we used a normal ResNet-18 or feed-forward backbone with a single
wide layer in the end. One limitation of training GPs is the lack of
scalability with respect to the number of inducing points. We use novel
computational techniques that allow us to train GPs with hundreds of thousands
of inducing points and with GPU acceleration. As shown in our experiments,
doing so has been essential to get a close match between the GPs and the ANNs
on 5 datasets. We implement our method as a publicly available tool called
GPEX: https://github.com/amirakbarnejad/gpex. On 5 datasets (4 image datasets,
and 1 biological dataset) and ANNs with 2 types of functionality (classifier or
attention-mechanism) we were able to find GPs whose outputs closely match those
of the corresponding ANNs. After matching the GPs to the ANNs, we used the GPs'
kernel functions to explain the ANNs' decisions. We provide more than 200
explanations (around 30 explanations in the paper and the rest in the
supplementary) which are highly interpretable by humans and show the ability of
the obtained GPs to unbox the ANNs' decisions.
