  Open-ended learning is a core research field of developmental robotics and AI
aiming to build learning machines and robots that can autonomously acquire
knowledge and skills incrementally as infants and children. The first
contribution of this work is to study the challenges posed by the previously
proposed benchmark `REAL competition' aiming to foster the development of truly
open-ended learning robot architectures. The competition involves a simulated
camera-arm robot that: (a) in a first `intrinsic phase' acquires sensorimotor
competence by autonomously interacting with objects; (b) in a second `extrinsic
phase' is tested with tasks unknown in the intrinsic phase to measure the
quality of knowledge previously acquired. This benchmark requires the solution
of multiple challenges usually tackled in isolation, in particular exploration,
sparse-rewards, object learning, generalisation, task/goal self-generation, and
autonomous skill learning. As a second contribution, we present a set of
`REAL-X' robot architectures that are able to solve different versions of the
benchmark, where we progressively release initial simplifications. The
architectures are based on a planning approach that dynamically increases
abstraction, and intrinsic motivations to foster exploration. REAL-X achieves a
good performance level in very demanding conditions. We argue that the REAL
benchmark represents a valuable tool for studying open-ended learning in its
hardest form.
