  Effectively exploring the environment is a key challenge in reinforcement
learning (RL). We address this challenge by defining a novel intrinsic reward
based on a foundation model, such as contrastive language image pretraining
(CLIP), which can encode a wealth of domain-independent semantic
visual-language knowledge about the world. Specifically, our intrinsic reward
is defined based on pre-trained CLIP embeddings without any fine-tuning or
learning on the target RL task. We demonstrate that CLIP-based intrinsic
rewards can drive exploration towards semantically meaningful states and
outperform state-of-the-art methods in challenging sparse-reward
procedurally-generated environments.
