  For better classification generative models are used to initialize the model
and model features before training a classifier. Typically it is needed to
solve separate unsupervised and supervised learning problems. Generative
restricted Boltzmann machines and deep belief networks are widely used for
unsupervised learning. We developed several supervised models based on DBN in
order to improve this two-phase strategy. Modifying the loss function to
account for expectation with respect to the underlying generative model,
introducing weight bounds, and multi-level programming are applied in model
development. The proposed models capture both unsupervised and supervised
objectives effectively. The computational study verifies that our models
perform better than the two-phase training approach.
