  We consider the problem of inferring the values of an arbitrary set of
variables (e.g., risk of diseases) given other observed variables (e.g.,
symptoms and diagnosed diseases) and high-dimensional signals (e.g., MRI images
or EEG). This is a common problem in healthcare since variables of interest
often differ for different patients. Existing methods including Bayesian
networks and structured prediction either do not incorporate high-dimensional
signals or fail to model conditional dependencies among variables. To address
these issues, we propose bidirectional inference networks (BIN), which stich
together multiple probabilistic neural networks, each modeling a conditional
dependency. Predictions are then made via iteratively updating variables using
backpropagation (BP) to maximize corresponding posterior probability.
Furthermore, we extend BIN to composite BIN (CBIN), which involves the
iterative prediction process in the training stage and improves both accuracy
and computational efficiency by adaptively smoothing the optimization
landscape. Experiments on synthetic and real-world datasets (a sleep study and
a dermatology dataset) show that CBIN is a single model that can achieve
state-of-the-art performance and obtain better accuracy in most inference tasks
than multiple models each specifically trained for a different task.
