  Let $A\subset [1,N]$ be a set of positive integers with $|A|\gg \sqrt N$. We
show that if avoids about $p/2$ residue classes modulo $p$ for each prime $p$,
the $A$ must correlate additively with the squares $S=\{n^2:1\leq n\leq \sqrt
N\}$, in the sense that we have the additive energy estimate $E(A,S)\gg N\log
N$. This is, in a sense, optimal.
