  A lot of recent machine learning research papers have ``open-ended learning''
in their title. But very few of them attempt to define what they mean when
using the term. Even worse, when looking more closely there seems to be no
consensus on what distinguishes open-ended learning from related concepts such
as continual learning, lifelong learning or autotelic learning. In this paper,
we contribute to fixing this situation. After illustrating the genealogy of the
concept and more recent perspectives about what it truly means, we outline that
open-ended learning is generally conceived as a composite notion encompassing a
set of diverse properties. In contrast with previous approaches, we propose to
isolate a key elementary property of open-ended processes, which is to produce
elements from time to time (e.g., observations, options, reward functions, and
goals), over an infinite horizon, that are considered novel from an observer's
perspective. From there, we build the notion of open-ended learning problems
and focus in particular on the subset of open-ended goal-conditioned
reinforcement learning problems in which agents can learn a growing repertoire
of goal-driven skills. Finally, we highlight the work that remains to be
performed to fill the gap between our elementary definition and the more
involved notions of open-ended learning that developmental AI researchers may
have in mind.
