  Model explainability is crucial for human users to be able to interpret how a
proposed classifier assigns labels to data based on its feature values. We
study generalized linear models constructed using sets of feature value rules,
which can capture nonlinear dependencies and interactions. An inherent
trade-off exists between rule set sparsity and its prediction accuracy. It is
computationally expensive to find the right choice of sparsity -- e.g., via
cross-validation -- with existing methods. We propose a new formulation to
learn an ensemble of rule sets that simultaneously addresses these competing
factors. Good generalization is ensured while keeping computational costs low
by utilizing distributionally robust optimization. The formulation utilizes
column generation to efficiently search the space of rule sets and constructs a
sparse ensemble of rule sets, in contrast with techniques like random forests
or boosting and their variants. We present theoretical results that motivate
and justify the use of our distributionally robust formulation. Extensive
numerical experiments establish that our method improves over competing methods
-- on a large set of publicly available binary classification problem instances
-- with respect to one or more of the following metrics: generalization
quality, computational cost, and explainability.
