  Recent work by Cohen \emph{et al.} has achieved state-of-the-art results for
learning spherical images in a rotation invariant way by using ideas from group
representation theory and noncommutative harmonic analysis. In this paper we
propose a generalization of this work that generally exhibits improved
performace, but from an implementation point of view is actually simpler. An
unusual feature of the proposed architecture is that it uses the
Clebsch--Gordan transform as its only source of nonlinearity, thus avoiding
repeated forward and backward Fourier transforms. The underlying ideas of the
paper generalize to constructing neural networks that are invariant to the
action of other compact groups.
