  In this paper we present a method which assigns to each layer of a multilayer
neural network, whose network dynamics is governed by a noisy winner-take-all
mechanism, a neural temperature. This neural temperature is obtained by a least
mean square fit of the probability distribution of the noisy winner-take-all
mechanism to the distribution of a softmax mechanism, which has a well defined
temperature as free parameter. We call this approximated temperature resulting
from the optimization step the neural temperature. We apply this method to a
multilayer neural network during learning the XOR-problem with a Hebb-like
learning rule and show that after a transient the neural temperature decreases
in each layer according to a power law. This indicates a self-organized
annealing behavior induced by the learning rule itself instead of an external
adjustment of a control parameter as in physically motivated optimization
methods e.g. simulated annealing.
