  The tremendous amount of accessible data in cyberspace face the risk of being
unauthorized used for training deep learning models. To address this concern,
methods are proposed to make data unlearnable for deep learning models by
adding a type of error-minimizing noise. However, such conferred unlearnability
is found fragile to adversarial training. In this paper, we design new methods
to generate robust unlearnable examples that are protected from adversarial
training. We first find that the vanilla error-minimizing noise, which
suppresses the informative knowledge of data via minimizing the corresponding
training loss, could not effectively minimize the adversarial training loss.
This explains the vulnerability of error-minimizing noise in adversarial
training. Based on the observation, robust error-minimizing noise is then
introduced to reduce the adversarial training loss. Experiments show that the
unlearnability brought by robust error-minimizing noise can effectively protect
data from adversarial training in various scenarios. The code is available at
\url{https://github.com/fshp971/robust-unlearnable-examples}.
