  We study conditions under which, given a dictionary $F=\{f_1,\ldots ,f_M\}$
and an i.i.d. sample $(X_i,Y_i)_{i=1}^N$, the empirical minimizer in
$\operatorname {span}(F)$ relative to the squared loss, satisfies that with
high probability
\[R\bigl(\tilde{f}^{\mathrm{ERM}}\bigr)\leq\inf_{f\in\operatorname
{span}(F)}R(f)+r_N(M),\] where $R(\cdot)$ is the squared risk and $r_N(M)$ is
of the order of $M/N$. Among other results, we prove that a uniform small-ball
estimate for functions in $\operatorname {span}(F)$ is enough to achieve that
goal when the noise is independent of the design.
