  In this work, we establish risk bounds for the Empirical Risk Minimization
(ERM) with both dependent and heavy-tailed data-generating processes. We do so
by extending the seminal works of Mendelson [Men15, Men18] on the analysis of
ERM with heavy-tailed but independent and identically distributed observations,
to the strictly stationary exponentially $\beta$-mixing case. Our analysis is
based on explicitly controlling the multiplier process arising from the
interaction between the noise and the function evaluations on inputs. It allows
for the interaction to be even polynomially heavy-tailed, which covers a
significantly large class of heavy-tailed models beyond what is analyzed in the
learning theory literature. We illustrate our results by deriving rates of
convergence for the high-dimensional linear regression problem with dependent
and heavy-tailed data.
