  The use and analysis of massive data are challenging due to the high storage
and computational cost. Subsampling algorithms are popular to downsize the data
volume and reduce the computational burden. Existing subsampling approaches
focus on data with numerical covariates. Although big data with categorical
covariates are frequently encountered in many disciplines, the subsampling plan
has not been well established. In this paper, we propose a balanced subsampling
approach for reducing data with categorical covariates. The selected subsample
achieves a combinatorial balance among values of covariates and therefore
enjoys three desired merits. First, a balanced subsample is nonsingular and
thus allows the estimation of all parameters in ANOVA regression. Second, it
provides the optimal parameter estimation in the sense of minimizing the
generalized variance of the estimated parameters. Third, the model trained on a
balanced subsample provides robust predictions in the sense of minimizing the
worst-case prediction error. We demonstrate the usefulness of the balanced
subsampling over existing data reduction methods in extensive simulation
studies and a real-world application.
