  How can agents learn internal models that veridically represent interactions
with the real world is a largely open question. As machine learning is moving
towards representations containing not just observational but also
interventional knowledge, we study this problem using tools from representation
learning and group theory. We propose methods enabling an agent acting upon the
world to learn internal representations of sensory information that are
consistent with actions that modify it. We use an autoencoder equipped with a
group representation acting on its latent space, trained using an
equivariance-derived loss in order to enforce a suitable homomorphism property
on the group representation. In contrast to existing work, our approach does
not require prior knowledge of the group and does not restrict the set of
actions the agent can perform. We motivate our method theoretically, and show
empirically that it can learn a group representation of the actions, thereby
capturing the structure of the set of transformations applied to the
environment. We further show that this allows agents to predict the effect of
sequences of future actions with improved accuracy.
