  We have formulated a family of machine learning problems as the time
evolution of Parametric Probabilistic Models (PPMs), inherently rendering a
thermodynamic process. Our primary motivation is to leverage the rich toolbox
of thermodynamics of information to assess the information-theoretic content of
learning a probabilistic model. We first introduce two information-theoretic
metrics: Memorized-information (M-info) and Learned-information (L-info), which
trace the flow of information during the learning process of PPMs. Then, we
demonstrate that the accumulation of L-info during the learning process is
associated with entropy production, and parameters serve as a heat reservoir in
this process, capturing learned information in the form of M-info.
