  We show the universality of depth-2 group convolutional neural networks
(GCNNs) in a unified and constructive manner based on the ridgelet theory.
Despite widespread use in applications, the approximation property of (G)CNNs
has not been well investigated. The universality of (G)CNNs has been shown
since the late 2010s. Yet, our understanding on how (G)CNNs represent functions
is incomplete because the past universality theorems have been shown in a
case-by-case manner by manually/carefully assigning the network parameters
depending on the variety of convolution layers, and in an indirect manner by
converting/modifying the (G)CNNs into other universal approximators such as
invariant polynomials and fully-connected networks. In this study, we formulate
a versatile depth-2 continuous GCNN $S[\gamma]$ as a nonlinear mapping between
group representations, and directly obtain an analysis operator, called the
ridgelet trasform, that maps a given function $f$ to the network parameter
$\gamma$ so that $S[\gamma]=f$. The proposed GCNN covers typical GCNNs such as
the cyclic convolution on multi-channel images, networks on
permutation-invariant inputs (Deep Sets), and $\mathrm{E}(n)$-equivariant
networks. The closed-form expression of the ridgelet transform can describe how
the network parameters are organized to represent a function. While it has been
known only for fully-connected networks, this study is the first to obtain the
ridgelet transform for GCNNs. By discretizing the closed-form expression, we
can systematically generate a constructive proof of the $cc$-universality of
finite GCNNs. In other words, our universality proofs are more unified and
constructive than previous proofs.
