  We propose a novel clustering mechanism based on an incompatibility property
between subsets of data that emerges during model training. This mechanism
partitions the dataset into subsets that generalize only to themselves, i.e.,
training on one subset does not improve performance on the other subsets.
Leveraging the interaction between the dataset and the training process, our
clustering mechanism partitions datasets into clusters that are defined by--and
therefore meaningful to--the objective of the training process.
  We apply our clustering mechanism to defend against data poisoning attacks,
in which the attacker injects malicious poisoned data into the training dataset
to affect the trained model's output. Our evaluation focuses on backdoor
attacks against deep neural networks trained to perform image classification
using the GTSRB and CIFAR-10 datasets. Our results show that (1) these attacks
produce poisoned datasets in which the poisoned and clean data are incompatible
and (2) our technique successfully identifies (and removes) the poisoned data.
In an end-to-end evaluation, our defense reduces the attack success rate to
below 1% on 134 out of 165 scenarios, with only a 2% drop in clean accuracy on
CIFAR-10 and a negligible drop in clean accuracy on GTSRB.
