  While using invariant and equivariant maps, it is possible to apply deep
learning to a range of primitive data structures, a formalism for dealing with
hierarchy is lacking. This is a significant issue because many practical
structures are hierarchies of simple building blocks; some examples include
sequences of sets, graphs of graphs, or multiresolution images. Observing that
the symmetry of a hierarchical structure is the "wreath product" of symmetries
of the building blocks, we express the equivariant map for the hierarchy using
an intuitive combination of the equivariant linear layers of the building
blocks. More generally, we show that any equivariant map for the hierarchy has
this form. To demonstrate the effectiveness of this approach to model design,
we consider its application in the semantic segmentation of point-cloud data.
By voxelizing the point cloud, we impose a hierarchy of translation and
permutation symmetries on the data and report state-of-the-art on Semantic3D,
S3DIS, and vKITTI, that include some of the largest real-world point-cloud
benchmarks.
