  Adversarial reprogramming, introduced by Elsayed, Goodfellow, and
Sohl-Dickstein, seeks to repurpose a neural network to perform a different
task, by manipulating its input without modifying its weights. We prove that
two-layer ReLU neural networks with random weights can be adversarially
reprogrammed to achieve arbitrarily high accuracy on Bernoulli data models over
hypercube vertices, provided the network width is no greater than its input
dimension. We also substantially strengthen a recent result of Phuong and
Lampert on directional convergence of gradient flow, and obtain as a corollary
that training two-layer ReLU neural networks on orthogonally separable datasets
can cause their adversarial reprogramming to fail. We support these theoretical
results by experiments that demonstrate that, as long as batch normalisation
layers are suitably initialised, even untrained networks with random weights
are susceptible to adversarial reprogramming. This is in contrast to
observations in several recent works that suggested that adversarial
reprogramming is not possible for untrained networks to any degree of
reliability.
