  We introduce a novel Information Criterion (IC), termed Learning under
Singularity (LS), designed to enhance the functionality of the Widely
Applicable Bayes Information Criterion (WBIC) and the Singular Bayesian
Information Criterion (sBIC). LS is effective without regularity constraints
and demonstrates stability. Watanabe defined a statistical model or a learning
machine as regular if the mapping from a parameter to a probability
distribution is one-to-one and its Fisher information matrix is positive
definite. In contrast, models not meeting these conditions are termed singular.
Over the past decade, several information criteria for singular cases have been
proposed, including WBIC and sBIC. WBIC is applicable in non-regular scenarios
but faces challenges with large sample sizes and redundant estimation of known
learning coefficients. Conversely, sBIC is limited in its broader application
due to its dependence on maximum likelihood estimates. LS addresses these
limitations by enhancing the utility of both WBIC and sBIC. It incorporates the
empirical loss from the Widely Applicable Information Criterion (WAIC) to
represent the goodness of fit to the statistical model, along with a penalty
term similar to that of sBIC. This approach offers a flexible and robust method
for model selection, free from regularity constraints.
