  The maximum likelihood method is the best-known method for estimating the
probabilities behind the data. However, the conventional method obtains the
probability model closest to the empirical distribution, resulting in
overfitting. Then regularization methods prevent the model from being
excessively close to the wrong probability, but little is known systematically
about their performance. The idea of regularization is similar to
error-correcting codes, which obtain optimal decoding by mixing suboptimal
solutions with an incorrectly received code. The optimal decoding in
error-correcting codes is achieved based on gauge symmetry. We propose a
theoretically guaranteed regularization in the maximum likelihood method by
focusing on a gauge symmetry in Kullback -- Leibler divergence. In our
approach, we obtain the optimal model without the need to search for
hyperparameters frequently appearing in regularization.
