  Training neural networks which are robust to adversarial attacks remains an
important problem in deep learning, especially as heavily overparameterized
models are adopted in safety-critical settings. Drawing from recent work which
reformulates the training problems for two-layer ReLU and polynomial activation
networks as convex programs, we devise a convex semidefinite program (SDP) for
adversarial training of polynomial activation networks via the S-procedure. We
also derive a convex SDP to compute the minimum distance from a correctly
classified example to the decision boundary of a polynomial activation network.
Adversarial training for two-layer ReLU activation networks has been explored
in the literature, but, in contrast to prior work, we present a scalable
approach which is compatible with standard machine libraries and GPU
acceleration. The adversarial training SDP for polynomial activation networks
leads to large increases in robust test accuracy against $\ell^\infty$ attacks
on the Breast Cancer Wisconsin dataset from the UCI Machine Learning
Repository. For two-layer ReLU networks, we leverage our scalable
implementation to retrain the final two fully connected layers of a
Pre-Activation ResNet-18 model on the CIFAR-10 dataset. Our 'robustified' model
achieves higher clean and robust test accuracies than the same architecture
trained with sharpness-aware minimization.
