  The current $3.8 \sigma$ tension between local (arXiv:1804.10655) and global
(arXiv:1605.02985) measurements of $H_0$ cannot be fully explained by the
concordance $\Lambda$CDM model. It could be produced by unknown systematics or
by physics beyond the standard model. On the other hand, it is well known that
linear perturbation theory predicts a cosmic variance on the Hubble parameter
$H_0$, which leads to systematic errors on its local determination. Here, we
study how including in the likelihood the cosmic variance on $H_0$ affects
statistical inference. In particular we consider the $\gamma$CDM, $w$CDM and
$\gamma w$CDM parametric extensions of the standard model, which we constrain
with the latest CMB, BAO, SNe Ia, RSD and $H_0$ data. We learn two important
lessons. First, the systematic error from cosmic variance is - independently of
the model - approximately $\sigma_{\text{cv}}\approx 0.88$ km s$^{-1}$
Mpc$^{-1}$ (1.2\% $H_0^{\text{loc}}$) when considering the redshift range
$0.0233 \le z \le 0.15$, which is relative to the main analysis of
(arXiv:1804.10655), and $\sigma_{\text{cv}}\approx 1.5$ km s$^{-1}$ Mpc$^{-1}$
(2.1\% $H_0^{\text{loc}}$) when considering the wider redshift range $0.01 \le
z \le 0.15$. Although $\sigma_{\text{cv}}$ affects the total error budget on
local $H_0$, it does not significantly alleviate the tension which remains at
$\approx 3 \sigma$. Second, cosmic variance, besides shifting the constraints,
can change the results of model selection: much of the statistical advantage of
non-standard models is to alleviate the now-reduced tension. We conclude that,
when constraining non-standard models it is important to include the cosmic
variance on $H_0$ if one wants to use the local determination of the Hubble
constant by Riess et al. (arXiv:1804.10655). Doing the contrary could
potentially bias the conclusions.
