  We develop a thermodynamic theory for machine learning (ML) systems. Similar
to physical thermodynamic systems which are characterized by energy and
entropy, ML systems possess these characteristics as well. This comparison
inspire us to integrate the concept of temperature into ML systems grounded in
the fundamental principles of thermodynamics, and establish a basic
thermodynamic framework for machine learning systems with non-Boltzmann
distributions. We introduce the concept of states within a ML system, identify
two typical types of state, and interpret model training and refresh as a
process of state phase transition. We consider that the initial potential
energy of a ML system is described by the model's loss functions, and the
energy adheres to the principle of minimum potential energy. For a variety of
energy forms and parameter initialization methods, we derive the temperature of
systems during the phase transition both analytically and asymptotically,
highlighting temperature as a vital indicator of system data distribution and
ML training complexity. Moreover, we perceive deep neural networks as complex
heat engines with both global temperature and local temperatures in each layer.
The concept of work efficiency is introduced within neural networks, which
mainly depends on the neural activation functions. We then classify neural
networks based on their work efficiency, and describe neural networks as two
types of heat engines.
