  This study demonstrates that double descent can be mitigated by adding a
dropout layer adjacent to the fully connected linear layer. The unexpected
double-descent phenomenon garnered substantial attention in recent years,
resulting in fluctuating prediction error rates as either sample size or model
size increases. Our paper posits that the optimal test error, in terms of the
dropout rate, shows a monotonic decrease in linear regression with increasing
sample size. Although we do not provide a precise mathematical proof of this
statement, we empirically validate through experiments that the test error
decreases for each dropout rate. The statement we prove is that the expected
test error for each dropout rate within a certain range decreases when the
dropout rate is fixed. Our experimental results substantiate our claim, showing
that dropout with an optimal dropout rate can yield a monotonic test error
curve in nonlinear neural networks. These experiments were conducted using the
Fashion-MNIST and CIFAR-10 datasets. These findings imply the potential benefit
of incorporating dropout into risk curve scaling to address the peak
phenomenon. To our knowledge, this study represents the first investigation
into the relationship between dropout and double descent.
