  We demonstrate an approach to face attribute detection that retains or
improves attribute detection accuracy across gender and race subgroups by
learning demographic information prior to learning the attribute detection
task. The system, which we call InclusiveFaceNet, detects face attributes by
transferring race and gender representations learned from a held-out dataset of
public race and gender identities. Leveraging learned demographic
representations while withholding demographic inference from the downstream
face attribute detection task preserves potential users' demographic privacy
while resulting in some of the best reported numbers to date on attribute
detection in the Faces of the World and CelebA datasets.
