  Recurrent Neural Networks (RNNs) have been a prominent concept within
artificial intelligence. They are inspired by Biological Neural Networks (BNNs)
and provide an intuitive and abstract representation of how BNNs work. Derived
from the more generic Artificial Neural Networks (ANNs), the recurrent ones are
meant to be used for temporal tasks, such as speech recognition, because they
are capable of memorizing historic input. However, such networks are very time
consuming to train as a result of their inherent nature. Recently, Echo State
Networks and Liquid State Machines have been proposed as possible RNN
alternatives, under the name of Reservoir Computing (RC). RCs are far more easy
to train. In this paper, Cellular Automata are used as reservoir, and are
tested on the 5-bit memory task (a well known benchmark within the RC
community). The work herein provides a method of mapping binary inputs from the
task onto the automata, and a recurrent architecture for handling the
sequential aspects of it. Furthermore, a layered (deep) reservoir architecture
is proposed. Performances are compared towards earlier work, in addition to its
single-layer version. Results show that the single CA reservoir system yields
similar results to state-of-the-art work. The system comprised of two layered
reservoirs do show a noticeable improvement compared to a single CA reservoir.
This indicates potential for further research and provides valuable insight on
how to design CA reservoir systems.
