  It was recently shown that almost all solutions in the symmetric binary
perceptron are isolated, even at low constraint densities, suggesting that
finding typical solutions is hard. In contrast, some algorithms have been shown
empirically to succeed in finding solutions at low density. This phenomenon has
been justified numerically by the existence of subdominant and dense connected
regions of solutions, which are accessible by simple learning algorithms. In
this paper, we establish formally such a phenomenon for both the symmetric and
asymmetric binary perceptrons. We show that at low constraint density
(equivalently for overparametrized perceptrons), there exists indeed a
subdominant connected cluster of solutions with almost maximal diameter, and
that an efficient multiscale majority algorithm can find solutions in such a
cluster with high probability, settling in particular an open problem posed by
Perkins-Xu '21. In addition, even close to the critical threshold, we show that
there exist clusters of linear diameter for the symmetric perceptron, as well
as for the asymmetric perceptron under additional assumptions.
