  The ever-growing size of the datasets renders well-studied learning
techniques, such as Kernel Ridge Regression, inapplicable, posing a serious
computational challenge. Divide-and-conquer is a common remedy, suggesting to
split the dataset into disjoint partitions, obtain the local estimates and
average them, it allows to scale-up an otherwise ineffective base approach. In
the current study we suggest a fully data-driven approach to quantify
uncertainty of the averaged estimator. Namely, we construct simultaneous
element-wise confidence bands for the predictions yielded by the averaged
estimator on a given deterministic prediction set. The novel approach features
rigorous theoretical guaranties for a wide class of base learners with Kernel
Ridge regression being a special case. As a by-product of our analysis we also
obtain a sup-norm consistency result for the divide-and-conquer Kernel Ridge
Regression. The simulation study supports the theoretical findings.
