  Random forest is widely exploited as an ensemble learning method. In many
practical applications, however, there is still a significant challenge to
learn from imbalanced data. To alleviate this limitation, we propose a deep
dynamic boosted forest (DDBF), a novel ensemble algorithm that incorporates the
notion of hard example mining into random forest. Specically, we propose to
measure the quality of each leaf node of every decision tree in the random
forest to determine hard examples. By iteratively training and then removing
easy examples from training data, we evolve the random forest to focus on hard
examples dynamically so as to balance the proportion of samples and learn
decision boundaries better. Data can be cascaded through these random forests
learned in each iteration in sequence to generate more accurate predictions.
Our DDBF outperforms random forest on 5 UCI datasets, MNIST and SATIMAGE, and
achieved state-of-the-art results compared to other deep models. Moreover, we
show that DDBF is also a new way of sampling and can be very useful and
efficient when learning from imbalanced data.
