  Biological systems have to build models from their sensory data that allow
them to efficiently process previously unseen inputs. Here, we study a neural
network learning a linearly separable rule using examples provided by a
teacher. We analyse the ability of the network to apply the rule to new inputs,
that is to generalise from past experience. Using stochastic thermodynamics, we
show that the thermodynamic costs of the learning process provide an upper
bound on the amount of information that the network is able to learn from its
teacher for both batch and online learning. This allows us to introduce a
thermodynamic efficiency of learning. We analytically compute the dynamics and
the efficiency of a noisy neural network performing online learning in the
thermodynamic limit. In particular, we analyse three popular learning
algorithms, namely Hebbian, Perceptron and AdaTron learning. Our work extends
the methods of stochastic thermodynamics to a new type of learning problem and
might form a suitable basis for investigating the thermodynamics of
decision-making.
