  Robustness to adversarial attacks is typically obtained through expensive
adversarial training with Projected Gradient Descent. Here we introduce ROPUST,
a remarkably simple and efficient method to leverage robust pre-trained models
and further increase their robustness, at no cost in natural accuracy. Our
technique relies on the use of an Optical Processing Unit (OPU), a photonic
co-processor, and a fine-tuning step performed with Direct Feedback Alignment,
a synthetic gradient training scheme. We test our method on nine different
models against four attacks in RobustBench, consistently improving over
state-of-the-art performance. We perform an ablation study on the single
components of our defense, showing that robustness arises from parameter
obfuscation and the alternative training method. We also introduce phase
retrieval attacks, specifically designed to increase the threat level of
attackers against our own defense. We show that even with state-of-the-art
phase retrieval techniques, ROPUST remains an effective defense.
