  Representation learning, especially which by using deep learning, has been
widely applied in classification. However, how to use limited size of labeled
data to achieve good classification performance with deep neural network, and
how can the learned features further improve classification remain indefinite.
In this paper, we propose Horizontal Voting Vertical Voting and Horizontal
Stacked Ensemble methods to improve the classification performance of deep
neural networks. In the ICML 2013 Black Box Challenge, via using these methods
independently, Bing Xu achieved 3rd in public leaderboard, and 7th in private
leaderboard; Jingjing Xie achieved 4th in public leaderboard, and 5th in
private leaderboard.
