  We prove an impossibility result, which in the context of function learning
says the following: under certain conditions, it is impossible to
simultaneously learn symmetries and functions equivariant under them using an
ansatz consisting of equivariant functions. To formalize this statement, we
carefully study notions of approximation for groups and semigroups. We analyze
certain families of neural networks for whether they satisfy the conditions of
the impossibility result: what we call ``linearly equivariant'' networks, and
group-convolutional networks. A lot can be said precisely about linearly
equivariant networks, making them theoretically useful. On the practical side,
our analysis of group-convolutional neural networks allows us generalize the
well-known ``convolution is all you need'' theorem to non-homogeneous spaces.
We additionally find an important difference between group convolution and
semigroup convolution.
