  Preprocessing defenses such as pixel discretization are appealing to remove
adversarial attacks due to their simplicity. However, they have been shown to
be ineffective except on simple datasets like MNIST. We hypothesize that
existing discretization approaches failed because using a fixed codebook for
the entire dataset limits their ability to balance image representation and
codeword separability. We first formally prove that adaptive codebooks can
provide stronger robustness guarantees than fixed codebooks as a preprocessing
defense on some datasets. Based on that insight, we propose a content-adaptive
pixel discretization defense called Essential Features, which discretizes the
image to a per-image adaptive codebook to reduce the color space. We then find
that Essential Features can be further optimized by applying adaptive blurring
before the discretization to push perturbed pixel values back to their original
value before determining the codebook. Against adaptive attacks, we show that
content-adaptive pixel discretization extends the range of datasets that
benefit in terms of both L_2 and L_infinity robustness where previously fixed
codebooks were found to have failed. Our findings suggest that content-adaptive
pixel discretization should be part of the repertoire for making models robust.
