  Increasing the size of overparameterized neural networks has been a key in
achieving state-of-the-art performance. This is captured by the double descent
phenomenon, where the test loss follows a decreasing-increasing-decreasing
pattern (or sometimes monotonically decreasing) as model width increases.
However, the effect of label noise on the test loss curve has not been fully
explored. In this work, we uncover an intriguing phenomenon where label noise
leads to a \textit{final ascent} in the originally observed double descent
curve. Specifically, under a sufficiently large noise-to-sample-size ratio,
optimal generalization is achieved at intermediate widths. Through theoretical
analysis, we attribute this phenomenon to the shape transition of test loss
variance induced by label noise. Furthermore, we extend the final ascent
phenomenon to model density and provide the first theoretical characterization
showing that reducing density by randomly dropping trainable parameters
improves generalization under label noise. We also thoroughly examine the roles
of regularization and sample size. Surprisingly, we find that larger $\ell_2$
regularization and robust learning methods against label noise exacerbate the
final ascent. We confirm the validity of our findings through extensive
experiments on ReLu networks trained on MNIST, ResNets/ViTs trained on
CIFAR-10/100, and InceptionResNet-v2 trained on Stanford Cars with real-world
noisy labels.
