  While deep learning has achieved remarkable success, there is no clear
explanation about why it works so well. In order to discuss this question
quantitatively, we need a mathematical framework that explains what learning is
in the first place. After several considerations, we succeeded in constructing
a mathematical framework that can provide a unified understanding of all types
of learning, including deep learning and learning in the brain. We call it
learning principle, and it follows that all learning is equivalent to
estimating the probability of input data. We not only derived this principle,
but also mentioned its application to actual machine learning models. For
example, we found that conventional supervised learning is equivalent to
estimating conditional probabilities, and succeeded in making supervised
learning more effective and generalized. We also proposed a new method of
defining the values of estimated probability using differentiation, and showed
that unsupervised learning can be performed on arbitrary dataset without any
prior knowledge. Namely, this method is a general-purpose machine learning in
the true sense. Moreover, we succeeded in describing the learning mechanism in
the brain by considering the time evolution of a fully or partially connected
model and applying this new method. The learning principle provides solutions
to many unsolved problems in deep learning and cognitive neuroscience.
