  We consider a new class of non Markovian processes with a countable number of
interacting components. At each time unit, each component can take two values,
indicating if it has a spike or not at this precise moment. The system evolves
as follows. For each component, the probability of having a spike at the next
time unit depends on the entire time evolution of the system after the last
spike time of the component. This class of systems extends in a non trivial way
both the interacting particle systems, which are Markovian, and the stochastic
chains with memory of variable length which have finite state space. These
features make it suitable to describe the time evolution of biological neural
systems. We construct a stationary version of the process by using a
probabilistic tool which is a Kalikow-type decomposition either in random
environment or in space-time. This construction implies uniqueness of the
stationary process. Finally we consider the case where the interactions between
components are given by a critical directed Erd\"os-R\'enyi-type random graph
with a large but finite number of components. In this framework we obtain an
explicit upper-bound for the correlation between successive inter-spike
intervals which is compatible with previous empirical findings.
