  Motivated by analyzing complicated and non-stationary time series, we study a
generalization of the scattering transform (ST) that includes broad neural
activation functions, which is called neural activation ST (NAST). On the
whole, NAST is a transform that comprises a sequence of ``neural processing
units'', each of which applies a high pass filter to the input from the
previous layer followed by a composition with a nonlinear function as the
output to the next neuron. Here, the nonlinear function models how a neuron
gets excited by the input signal. In addition to showing properties like
non-expansion, horizontal translational invariability and insensitivity to
local deformation, the statistical properties of the second order NAST of a
Gaussian process with various dependence and (non-)stationarity structure and
its interaction with the chosen high pass filters and activation functions are
explored and central limit theorem (CLT) and non-CLT results are provided.
Numerical simulations are also provided. The results explain how NAST processes
complicated and non-stationary time series, and pave a way towards statistical
inference based on NAST under the non-null case.
