  It is widely believed that engineering a model to be invariant/equivariant
improves generalisation. Despite the growing popularity of this approach, a
precise characterisation of the generalisation benefit is lacking. By
considering the simplest case of linear models, this paper provides the first
provably non-zero improvement in generalisation for invariant/equivariant
models when the target distribution is invariant/equivariant with respect to a
compact group. Moreover, our work reveals an interesting relationship between
generalisation, the number of training examples and properties of the group
action. Our results rest on an observation of the structure of function spaces
under averaging operators which, along with its consequences for feature
averaging, may be of independent interest.
