  This paper is motivated by the following question in sieve theory. Given a
subset $X\subset [N]$ and $\alpha\in (0,1/2)$. Suppose that $|X\pmod p|\leq
(\alpha+o(1))p$ for every prime $p$. How large can $X$ be? On the one hand, we
have the bound $|X|\ll_{\alpha}N^{\alpha}$ from Gallagher's larger sieve. On
the other hand, we prove, assuming the truth of an inverse sieve conjecture,
that the bound above can be improved (for example, to
$|X|\ll_{\alpha}N^{O(\alpha^{2014})}$ for small $\alpha$). The result follows
from studying the average size of $|X\pmod p|$ as $p$ varies, when
$X=f(\mathbb{Z})\cap [N]$ is the value set of a polynomial
$f(x)\in\mathbb{Z}[x]$.
