  Learning data representations that capture task-related features, but are
invariant to nuisance variations remains a key challenge in machine learning.
We introduce an automated Bayesian inference framework, called AutoBayes, that
explores different graphical models linking classifier, encoder, decoder,
estimator and adversarial network blocks to optimize nuisance-invariant machine
learning pipelines. AutoBayes also enables learning disentangled
representations, where the latent variable is split into multiple pieces to
impose various relationships with the nuisance variation and task labels. We
benchmark the framework on several public datasets, and provide analysis of its
capability for subject-transfer learning with/without variational modeling and
adversarial training. We demonstrate a significant performance improvement with
ensemble learning across explored graphical models.
