  Despite their success deep neural networks (DNNs) are still largely
considered as black boxes. The main issue is that the linear and non-linear
operations are entangled in every layer, making it hard to interpret the hidden
layer outputs. In this paper, we look at DNNs with rectified linear units
(ReLUs), and focus on the gating property (`on/off' states) of the ReLUs. We
extend the recently developed dual view in which the computation is broken
path-wise to show that learning in the gates is more crucial, and learning the
weights given the gates is characterised analytically via the so called neural
path kernel (NPK) which depends on inputs and gates. In this paper, we present
novel results to show that convolution with global pooling and skip connection
provide respectively rotational invariance and ensemble structure to the NPK.
To address `black box'-ness, we propose a novel interpretable counterpart of
DNNs with ReLUs namely deep linearly gated networks (DLGN): the pre-activations
to the gates are generated by a deep linear network, and the gates are then
applied as external masks to learn the weights in a different network. The DLGN
is not an alternative architecture per se, but a disentanglement and an
interpretable re-arrangement of the computations in a DNN with ReLUs. The DLGN
disentangles the computations into two `mathematically' interpretable
linearities (i) the `primal' linearity between the input and the
pre-activations in the gating network and (ii) the `dual' linearity in the path
space in the weights network characterised by the NPK. We compare the
performance of DNN, DGN and DLGN on CIFAR-10 and CIFAR-100 to show that, the
DLGN recovers more than $83.5\%$ of the performance of state-of-the-art DNNs.
This brings us to an interesting question: `Is DLGN a universal spectral
approximator?'
