  State-of-the-art machine learning models can be vulnerable to very small
input perturbations that are adversarially constructed. Adversarial training is
an effective approach to defend against such examples. It is formulated as a
min-max problem, searching for the best solution when the training data was
corrupted by the worst-case attacks. For linear regression problems,
adversarial training can be formulated as a convex problem. We use this
reformulation to make two technical contributions: First, we formulate the
training problem as an instance of robust regression to reveal its connection
to parameter-shrinking methods, specifically that $\ell_\infty$-adversarial
training produces sparse solutions. Secondly, we study adversarial training in
the overparameterized regime, i.e. when there are more parameters than data. We
prove that adversarial training with small disturbances gives the solution with
the minimum-norm that interpolates the training data. Ridge regression and
lasso approximate such interpolating solutions as their regularization
parameter vanishes. By contrast, for adversarial training, the transition into
the interpolation regime is abrupt and for non-zero values of disturbance. This
result is proved and illustrated with numerical examples.
