  Braking index measurements of young radio pulsars are all smaller than the
value expected for spin down by magnetic dipole braking. We investigate
magnetic field evolution in the neutron star crust due to Hall drift as an
explanation for observed braking indices. Using numerical simulations and a
semi-analytic model, we show that a $\approx 10^{14}\ {\rm G}$ quadrupolar
toroidal field in the neutron star crust at birth leads to growth of the dipole
moment at a rate large enough to agree with measured braking indices. A key
factor is the density at which the crust yields to magnetic stresses that build
up during the evolution, which sets a characteristic minimum Hall timescale.
The observed braking indices of pulsars with inferred dipole fields of
$\lesssim 10^{13}\ {\rm G}$ can be explained in this picture, although with a
significant octupole component needed in some cases. For the stronger field
pulsars, those with $B_d\gtrsim 10^{13}\ {\rm G}$, we find that the magnetic
stresses in the crust exceed the maximum shear stress before the pulsar reaches
its current age, likely quenching the Hall effect. This may have implications
for the magnetar activity seen in the high magnetic field radio pulsar
PSR~J1846-0258. Observations of braking indices may therefore be a new piece of
evidence that neutron stars contain subsurface toroidal fields that are
significantly stronger than the dipole field, and may indicate that the Hall
effect is important in a wider range of neutron stars than previously thought.
