  Quality Diversity (QD) has shown great success in discovering
high-performing, diverse policies for robot skill learning. While current
benchmarks have led to the development of powerful QD methods, we argue that
new paradigms must be developed to facilitate open-ended search and
generalizability. In particular, many methods focus on learning diverse agents
that each move to a different xy position in MAP-Elites-style bounded archives.
Here, we show that such tasks can be accomplished with a single,
goal-conditioned policy paired with a classical planner, achieving O(1) space
complexity w.r.t. the number of policies and generalization to task variants.
We hypothesize that this approach is successful because it extracts
task-invariant structural knowledge by modeling a relational graph between
adjacent cells in the archive. We motivate this view with emerging evidence
from computational neuroscience and explore connections between QD and models
of cognitive maps in human and other animal brains. We conclude with a
discussion exploring the relationships between QD and cognitive maps, and
propose future research directions inspired by cognitive maps towards future
generalizable algorithms capable of truly open-ended search.
