  Mixup is a regularization technique that artificially produces new samples
using convex combinations of original training points. This simple technique
has shown strong empirical performance, and has been heavily used as part of
semi-supervised learning techniques such as
mixmatch~\citep{berthelot2019mixmatch} and interpolation consistent training
(ICT)~\citep{verma2019interpolation}. In this paper, we look at Mixup through a
\emph{representation learning} lens in a semi-supervised learning setup. In
particular, we study the role of Mixup in promoting linearity in the learned
network representations. Towards this, we study two questions: (1) how does the
Mixup loss that enforces linearity in the \emph{last} network layer propagate
the linearity to the \emph{earlier} layers?; and (2) how does the enforcement
of stronger Mixup loss on more than two data points affect the convergence of
training? We empirically investigate these properties of Mixup on vision
datasets such as CIFAR-10, CIFAR-100 and SVHN. Our results show that supervised
Mixup training does not make \emph{all} the network layers linear; in fact the
\emph{intermediate layers} become more non-linear during Mixup training
compared to a network that is trained \emph{without} Mixup. However, when Mixup
is used as an unsupervised loss, we observe that all the network layers become
more linear resulting in faster training convergence.
