  The bootstrap provides a simple and powerful means of assessing the quality
of estimators. However, in settings involving large datasets, the computation
of bootstrap-based quantities can be prohibitively demanding. As an
alternative, we present the Bag of Little Bootstraps (BLB), a new procedure
which incorporates features of both the bootstrap and subsampling to obtain a
robust, computationally efficient means of assessing estimator quality. BLB is
well suited to modern parallel and distributed computing architectures and
retains the generic applicability, statistical efficiency, and favorable
theoretical properties of the bootstrap. We provide the results of an extensive
empirical and theoretical investigation of BLB's behavior, including a study of
its statistical correctness, its large-scale implementation and performance,
selection of hyperparameters, and performance on real data.
