  During recent years the interest of optimization and machine learning
communities in high-probability convergence of stochastic optimization methods
has been growing. One of the main reasons for this is that high-probability
complexity bounds are more accurate and less studied than in-expectation ones.
However, SOTA high-probability non-asymptotic convergence results are derived
under strong assumptions such as the boundedness of the gradient noise variance
or of the objective's gradient itself. In this paper, we propose several
algorithms with high-probability convergence results under less restrictive
assumptions. In particular, we derive new high-probability convergence results
under the assumption that the gradient/operator noise has bounded central
$\alpha$-th moment for $\alpha \in (1,2]$ in the following setups: (i) smooth
non-convex / Polyak-Lojasiewicz / convex / strongly convex / quasi-strongly
convex minimization problems, (ii) Lipschitz / star-cocoercive and monotone /
quasi-strongly monotone variational inequalities. These results justify the
usage of the considered methods for solving problems that do not fit standard
functional classes studied in stochastic optimization.
