  Go-Explore is a powerful family of algorithms designed to solve
hard-exploration problems, built on the principle of archiving discovered
states, and iteratively returning to and exploring from the most promising
states. This approach has led to superhuman performance across a wide variety
of challenging problems including Atari games and robotic control, but requires
manually designing heuristics to guide exploration, which is time-consuming and
infeasible in general. To resolve this, we propose Intelligent Go-Explore (IGE)
which greatly extends the scope of the original Go-Explore by replacing these
heuristics with the intelligence and internalized human notions of
interestingness captured by giant foundation models (FMs). This provides IGE
with a human-like ability to instinctively identify how interesting or
promising any new state is (e.g. discovering new objects, locations, or
behaviors), even in complex environments where heuristics are hard to define.
Moreover, IGE offers the exciting and previously impossible opportunity to
recognize and capitalize on serendipitous discoveries that cannot be predicted
ahead of time. We evaluate IGE on a range of language-based tasks that require
search and exploration. In Game of 24, a multistep mathematical reasoning
problem, IGE reaches 100% success rate 70.8% faster than the best classic graph
search baseline. Next, in BabyAI-Text, a challenging partially observable
gridworld, IGE exceeds the previous SOTA with orders of magnitude fewer online
samples. Finally, in TextWorld, we show the unique ability of IGE to succeed in
settings requiring long-horizon exploration where prior SOTA FM agents like
Reflexion completely fail. Overall, IGE combines the tremendous strengths of
FMs and the powerful Go-Explore algorithm, opening up a new frontier of
research into creating more generally capable agents with impressive
exploration capabilities.
