  Randomized smoothing has achieved state-of-the-art certified robustness
against $l_2$-norm adversarial attacks. However, it is not wholly resolved on
how to find the optimal base classifier for randomized smoothing. In this work,
we employ a Smoothed WEighted ENsembling (SWEEN) scheme to improve the
performance of randomized smoothed classifiers. We show the ensembling
generality that SWEEN can help achieve optimal certified robustness.
Furthermore, theoretical analysis proves that the optimal SWEEN model can be
obtained from training under mild assumptions. We also develop an adaptive
prediction algorithm to reduce the prediction and certification cost of SWEEN
models. Extensive experiments show that SWEEN models outperform the upper
envelope of their corresponding candidate models by a large margin. Moreover,
SWEEN models constructed using a few small models can achieve comparable
performance to a single large model with a notable reduction in training time.
