  Learning disentangled representations that correspond to factors of variation
in real-world data is critical to interpretable and human-controllable machine
learning. Recently, concerns about the viability of learning disentangled
representations in a purely unsupervised manner has spurred a shift toward the
incorporation of weak supervision. However, there is currently no formalism
that identifies when and how weak supervision will guarantee disentanglement.
To address this issue, we provide a theoretical framework to assist in
analyzing the disentanglement guarantees (or lack thereof) conferred by weak
supervision when coupled with learning algorithms based on distribution
matching. We empirically verify the guarantees and limitations of several weak
supervision methods (restricted labeling, match-pairing, and rank-pairing),
demonstrating the predictive power and usefulness of our theoretical framework.
