  Recent advances in center-based clustering continue to improve upon the
drawbacks of Lloyd's celebrated $k$-means algorithm over $60$ years after its
introduction. Various methods seek to address poor local minima, sensitivity to
outliers, and data that are not well-suited to Euclidean measures of fit, but
many are supported largely empirically. Moreover, combining such approaches in
a piecemeal manner can result in ad hoc methods, and the limited theoretical
results supporting each individual contribution may no longer hold. Toward
addressing these issues in a principled way, this paper proposes a cohesive
robust framework for center-based clustering under a general class of
dissimilarity measures. In particular, we present a rigorous theoretical
treatment within a Median-of-Means (MoM) estimation framework, showing that it
subsumes several popular $k$-means variants. In addition to unifying existing
methods, we derive uniform concentration bounds that complete their analyses,
and bridge these results to the MoM framework via Dudley's chaining arguments.
Importantly, we neither require any assumptions on the distribution of the
outlying observations nor on the relative number of observations $n$ to
features $p$. We establish strong consistency and an error rate of
$O(n^{-1/2})$ under mild conditions, surpassing the best-known results in the
literature. The methods are empirically validated thoroughly on real and
synthetic datasets.
