  Using an approximation by a set-valued dynamical system, this paper studies a
class of non Markovian and non homogeneous stochastic processes on a finite
state space. It provides an unified approach to simulated annealing type
processes. It permits to study new models of vertex reinforced random walks and
new models of learning in games including Markovian fictitious play.
