  Echo State Networks (ESN) are a class of Recurrent Neural Networks (RNN) that
has gained substantial popularity due to their effectiveness, ease of use and
potential for compact hardware implementation. An ESN contains the three
network layers input, reservoir and readout where the reservoir is the truly
recurrent network. The input and reservoir layers of an ESN are initialized at
random and never trained afterwards and the training of the ESN is applied to
the readout layer only. The alternative of Recursive Neural Gas (RNG) is one of
the many proposals of fully-trainable reservoirs that can be found in the
literature. Although some improvements in performance have been reported with
RNG, to the best of authors' knowledge, no experimental comparative results are
known with benchmarks for which ESN is known to yield excellent results. This
work describes an accurate model of RNG together with some extensions to the
models presented in the literature and shows comparative results on three
well-known and accepted datasets. The experimental results obtained show that,
under specific circumstances, RNG-based reservoirs can achieve better
performance.
