  This paper introduces a new method for model selection and more generally
hyperparameter selection in machine learning. Minimum description length (MDL)
is an established method for model selection, which is however not directly
aimed at minimizing generalization error, which is often the primary goal in
machine learning. The paper demonstrates a relationship between generalization
error and a difference of description lengths of the training data; we call
this difference differential description length (DDL). This allows prediction
of generalization error from the training data alone by performing encoding of
the training data. DDL can then be used for model selection by choosing the
model with the smallest predicted generalization error. We show how this method
can be used for linear regression and neural networks and deep learning.
Experimental results show that DDL leads to smaller generalization error than
cross-validation and traditional MDL and Bayes methods.
