  Recent researches have shown that deep forest ensemble achieves a
considerable increase in classification accuracy compared with the general
ensemble learning methods, especially when the training set is small. In this
paper, we take advantage of deep forest ensemble and introduce the Dense
Adaptive Cascade Forest (daForest). Our model has a better performance than the
original Cascade Forest with three major features: first, we apply SAMME.R
boosting algorithm to improve the performance of the model. It guarantees the
improvement as the number of layers increases. Second, our model connects each
layer to the subsequent ones in a feed-forward fashion, which enhances the
capability of the model to resist performance degeneration. Third, we add a
hyper-parameters optimization layer before the first classification layer,
making our model spend less time to set up and find the optimal
hyper-parameters. Experimental results show that daForest performs
significantly well, and in some cases, even outperforms neural networks and
achieves state-of-the-art results.
