  This paper proposes a method to construct pretext tasks for self-supervised
learning on group equivariant neural networks. Group equivariant neural
networks are the models whose structure is restricted to commute with the
transformations on the input. Therefore, it is important to construct pretext
tasks for self-supervised learning that do not contradict this equivariance. To
ensure that training is consistent with the equivariance, we propose two
concepts for self-supervised tasks: equivariant pretext labels and invariant
contrastive loss. Equivariant pretext labels use a set of labels on which we
can define the transformations that correspond to the input change. Invariant
contrastive loss uses a modified contrastive loss that absorbs the effect of
transformations on each input. Experiments on standard image recognition
benchmarks demonstrate that the equivariant neural networks exploit the
proposed equivariant self-supervised tasks.
