  We derive and study time-uniform confidence spheres -- confidence sphere
sequences (CSSs) -- which contain the mean of random vectors with high
probability simultaneously across all sample sizes. Inspired by the original
work of Catoni and Giulini, we unify and extend their analysis to cover both
the sequential setting and to handle a variety of distributional assumptions.
Our results include an empirical-Bernstein CSS for bounded random vectors
(resulting in a novel empirical-Bernstein confidence interval with asymptotic
width scaling proportionally to the true unknown variance), CSSs for sub-$\psi$
random vectors (which includes sub-gamma, sub-Poisson, and sub-exponential),
and CSSs for heavy-tailed random vectors (two moments only). Finally, we
provide two CSSs that are robust to contamination by Huber noise. The first is
a robust version of our empirical-Bernstein CSS, and the second extends recent
work in the univariate setting to heavy-tailed multivariate distributions.
