  Equivariance of linear neural network layers is well studied. In this work,
we relax the equivariance condition to only be true in a projective sense. We
propose a way to construct a projectively equivariant neural network through
building a standard equivariant network where the linear group representations
acting on each intermediate feature space are "multiplicatively modified lifts"
of projective group representations. By theoretically studying the relation of
projectively and linearly equivariant linear layers, we show that our approach
is the most general possible when building a network out of linear layers. The
theory is showcased in two simple experiments.
