  We present new algorithms for online convex optimization over unbounded
domains that obtain parameter-free regret in high-probability given access only
to potentially heavy-tailed subgradient estimates. Previous work in unbounded
domains considers only in-expectation results for sub-exponential subgradients.
Unlike in the bounded domain case, we cannot rely on straight-forward
martingale concentration due to exponentially large iterates produced by the
algorithm. We develop new regularization techniques to overcome these problems.
Overall, with probability at most $\delta$, for all comparators $\mathbf{u}$
our algorithm achieves regret $\tilde{O}(\| \mathbf{u} \| T^{1/\mathfrak{p}}
\log (1/\delta))$ for subgradients with bounded $\mathfrak{p}^{th}$ moments for
some $\mathfrak{p} \in (1, 2]$.
