  We present Collaborative Trees, a novel tree model designed for regression
prediction, along with its bagging version, which aims to analyze complex
statistical associations between features and uncover potential patterns
inherent in the data. We decompose the mean decrease in impurity from the
proposed tree model to analyze the additive and interaction effects of features
on the response variable. Additionally, we introduce network diagrams to
visually depict how each feature contributes additively to the response and how
pairs of features contribute interaction effects. Through a detailed
demonstration using an embryo growth dataset, we illustrate how the new
statistical tools aid data analysis, both visually and numerically. Moreover,
we delve into critical aspects of tree modeling, such as prediction
performance, inference stability, and bias in feature importance measures,
leveraging real datasets and simulation experiments for comprehensive
discussions. On the theory side, we show that Collaborative Trees, built upon a
``sum of trees'' approach with our own innovative tree model regularization,
exhibit characteristics akin to matching pursuit, under the assumption of
high-dimensional independent binary input features (or one-hot feature groups).
This newfound link sheds light on the superior capability of our tree model in
estimating additive effects of features, a crucial factor for accurate
interaction effect estimation.
