  We study the problem of learning representations that are private yet
informative, i.e., provide information about intended "ally" targets while
hiding sensitive "adversary" attributes. We propose Exclusion-Inclusion
Generative Adversarial Network (EIGAN), a generalized private representation
learning (PRL) architecture that accounts for multiple ally and adversary
attributes unlike existing PRL solutions. While centrally-aggregated dataset is
a prerequisite for most PRL techniques, data in real-world is often siloed
across multiple distributed nodes unwilling to share the raw data because of
privacy concerns. We address this practical constraint by developing D-EIGAN,
the first distributed PRL method that learns representations at each node
without transmitting the source data. We theoretically analyze the behavior of
adversaries under the optimal EIGAN and D-EIGAN encoders and the impact of
dependencies among ally and adversary tasks on the optimization objective. Our
experiments on various datasets demonstrate the advantages of EIGAN in terms of
performance, robustness, and scalability. In particular, EIGAN outperforms the
previous state-of-the-art by a significant accuracy margin (47% improvement),
and D-EIGAN's performance is consistently on par with EIGAN under different
network settings.
