  Privacy-preserving instance encoding aims to encode raw data as feature
vectors without revealing their privacy-sensitive information. When designed
properly, these encodings can be used for downstream ML applications such as
training and inference with limited privacy risk. However, the vast majority of
existing instance encoding schemes are based on heuristics and their
privacy-preserving properties are only validated empirically against a limited
set of attacks. In this paper, we propose a theoretically-principled measure
for the privacy of instance encoding based on Fisher information. We show that
our privacy measure is intuitive, easily applicable, and can be used to bound
the invertibility of encodings both theoretically and empirically.
