  The lack of interpretability and trust is a much-criticised feature of deep
neural networks. In fully connected nets, the signalling between inner layers
is scrambled because backpropagation training does not require perceptrons to
be arranged in any particular order. The result is a black box; this problem is
particularly severe in scientific computing and digital signal processing
(DSP), where neutral nets perform abstract mathematical transformations that do
not reduce to features or concepts.
  We present here a group-theoretical procedure that attempts to bring inner
layer signalling into a human-readable form, the assumption being that this
form exists and has identifiable and quantifiable features - for example,
smoothness or locality. We applied the proposed method to DEERNet (a DSP
network used in electron spin resonance) and managed to descramble it. We found
considerable internal sophistication: the network spontaneously invents a
bandpass filter, a notch filter, a frequency axis rectifier, frequency division
multiplexing, group embedding, spectral filtering regularisation, and a
Fourier-like transform that maps harmonic functions into Chebyshev polynomials
- in ten minutes of unattended training from a random initial guess.
