  Recent work by Lakshminarayanan and Singh [2020] provided a dual view for
fully connected deep neural networks (DNNs) with rectified linear units (ReLU).
It was shown that (i) the information in the gates is analytically
characterised by a kernel called the neural path kernel (NPK) and (ii) most
critical information is learnt in the gates, in that, given the learnt gates,
the weights can be retrained from scratch without significant loss in
performance. Using the dual view, in this paper, we rethink the conventional
interpretations of DNNs thereby explicitsing the implicit interpretability of
DNNs. Towards this, we first show new theoretical properties namely rotational
invariance and ensemble structure of the NPK in the presence of convolutional
layers and skip connections respectively. Our theory leads to two surprising
empirical results that challenge conventional wisdom: (i) the weights can be
trained even with a constant 1 input, (ii) the gating masks can be shuffled,
without any significant loss in performance. These results motivate a novel
class of networks which we call deep linearly gated networks (DLGNs). DLGNs
using the phenomenon of dual lifting pave way to more direct and simpler
interpretation of DNNs as opposed to conventional interpretations. We show via
extensive experiments on CIFAR-10 and CIFAR-100 that these DLGNs lead to much
better interpretability-accuracy tradeoff.
