  We propose a new ensemble framework for supervised learning, called machine
collaboration (MaC), using a collection of base machines for prediction tasks.
Unlike bagging/stacking (a parallel & independent framework) and boosting (a
sequential & top-down framework), MaC is a type of circular & interactive
learning framework. The circular & interactive feature helps the base machines
to transfer information circularly and update their structures and parameters
accordingly. The theoretical result on the risk bound of the estimator from MaC
reveals that the circular & interactive feature can help MaC reduce risk via a
parsimonious ensemble. We conduct extensive experiments on MaC using both
simulated data and 119 benchmark real datasets. The results demonstrate that in
most cases, MaC performs significantly better than several other
state-of-the-art methods, including classification and regression trees, neural
networks, stacking, and boosting.
