  Sabre is a defense to adversarial examples that was accepted at IEEE S&P
2024. We first reveal significant flaws in the evaluation that point to clear
signs of gradient masking. We then show the cause of this gradient masking: a
bug in the original evaluation code. By fixing a single line of code in the
original repository, we reduce Sabre's robust accuracy to 0%. In response to
this, the authors modify the defense and introduce a new defense component not
described in the original paper. But this fix contains a second bug; modifying
one more line of code reduces robust accuracy to below baseline levels. After
we released the first version of our paper online, the authors introduced
another change to the defense; by commenting out one line of code during attack
we reduce the robust accuracy to 0% again.
