  Recently, several studies consider the stochastic optimization problem but in
a heavy-tailed noise regime, i.e., the difference between the stochastic
gradient and the true gradient is assumed to have a finite $p$-th moment (say
being upper bounded by $\sigma^{p}$ for some $\sigma\geq0$) where $p\in(1,2]$,
which not only generalizes the traditional finite variance assumption ($p=2$)
but also has been observed in practice for several different tasks. Under this
challenging assumption, lots of new progress has been made for either convex or
nonconvex problems, however, most of which only consider smooth objectives. In
contrast, people have not fully explored and well understood this problem when
functions are nonsmooth. This paper aims to fill this crucial gap by providing
a comprehensive analysis of stochastic nonsmooth convex optimization with
heavy-tailed noises. We revisit a simple clipping-based algorithm, whereas,
which is only proved to converge in expectation but under the additional strong
convexity assumption. Under appropriate choices of parameters, for both convex
and strongly convex functions, we not only establish the first high-probability
rates but also give refined in-expectation bounds compared with existing works.
Remarkably, all of our results are optimal (or nearly optimal up to logarithmic
factors) with respect to the time horizon $T$ even when $T$ is unknown in
advance. Additionally, we show how to make the algorithm parameter-free with
respect to $\sigma$, in other words, the algorithm can still guarantee
convergence without any prior knowledge of $\sigma$. Furthermore, an initial
distance adaptive convergence rate is provided if $\sigma$ is assumed to be
known.
