  Achieving transparency in black-box deep learning algorithms is still an open
challenge. High dimensional features and decisions given by deep neural
networks (NN) require new algorithms and methods to expose its mechanisms.
Current state-of-the-art NN interpretation methods (e.g. Saliency maps,
DeepLIFT, LIME, etc.) focus more on the direct relationship between NN outputs
and inputs rather than the NN structure and operations itself. In current deep
NN operations, there is uncertainty over the exact role played by neurons with
fixed activation functions. In this paper, we achieve partially explainable
learning model by symbolically explaining the role of activation functions (AF)
under a scalable topology. This is carried out by modeling the AFs as adaptive
Gaussian Processes (GP), which sit within a novel scalable NN topology, based
on the Kolmogorov-Arnold Superposition Theorem (KST). In this scalable NN
architecture, the AFs are generated by GP interpolation between control points
and can thus be tuned during the back-propagation procedure via gradient
descent. The control points act as the core enabler to both local and global
adjustability of AF, where the GP interpolation constrains the intrinsic
autocorrelation to avoid over-fitting. We show that there exists a trade-off
between the NN's expressive power and interpretation complexity, under linear
KST topology scaling. To demonstrate this, we perform a case study on a binary
classification dataset of banknote authentication. By quantitatively and
qualitatively investigating the mapping relationship between inputs and output,
our explainable model can provide interpretation over each of the
one-dimensional attributes. These early results suggest that our model has the
potential to act as the final interpretation layer for deep neural networks.
