  Adversarial perturbation plays a significant role in the field of adversarial
robustness, which solves a maximization problem over the input data. We show
that the backward propagation of such optimization can accelerate $2\times$
(and thus the overall optimization including the forward propagation can
accelerate $1.5\times$), without any utility drop, if we only compute the
output gradient but not the parameter gradient during the backward propagation.
