  Prior work found that superhuman Go AIs like KataGo can be defeated by simple
adversarial strategies. In this paper, we study if simple defenses can improve
KataGo's worst-case performance. We test three natural defenses: adversarial
training on hand-constructed positions, iterated adversarial training, and
changing the network architecture. We find that some of these defenses are able
to protect against previously discovered attacks. Unfortunately, we also find
that none of these defenses are able to withstand adaptive attacks. In
particular, we are able to train new adversaries that reliably defeat our
defended agents by causing them to blunder in ways humans would not. Our
results suggest that building robust AI systems is challenging even in narrow
domains such as Go. For interactive examples of attacks and a link to our
codebase, see https://goattack.far.ai.
