  To deal with various datasets over different complexity, this paper presents
an self-adaptive learning model that combines the proposed Dynamic Connected
Neural Decision Networks (DNDN) and a new pruning method--Dynamic Soft Pruning
(DSP). DNDN is a combination of random forests and deep neural networks that
enjoys both the advantages of strong classification capability of tree-like
structure and representation learning capability of network structure. Based on
Deep Neural Decision Forests (DNDF), this paper adopts an end-to-end training
approach by representing the classification distribution with multiple randomly
initialized softmax layers, which further allows an ensemble of multiple random
forests attached to layers of neural network with different depth. We also
propose a soft pruning method DSP to reduce the redundant connections of the
network adaptively to avoid over-fitting simple dataset. The model demonstrates
no performance loss compared with unpruned models and even higher robustness
over different data and feature distribution. Extensive experiments on
different datasets demonstrate the superiority of the proposed model over other
popular algorithms in solving classification tasks.
