  One of the essential missions in the AI research community is to build an
autonomous embodied agent that can achieve high-level performance across a wide
spectrum of tasks. However, acquiring or manually designing rewards for all
open-ended tasks is unrealistic. In this paper, we propose a novel cross-modal
contrastive learning framework architecture, CLIP4MC, aiming to learn a
reinforcement learning (RL) friendly vision-language model (VLM) that serves as
an intrinsic reward function for open-ended tasks. Simply utilizing the
similarity between the video snippet and the language prompt is not RL-friendly
since standard VLMs may only capture the similarity at a coarse level. To
achieve RL-friendliness, we incorporate the task completion degree into the VLM
training objective, as this information can assist agents in distinguishing the
importance between different states. Moreover, we provide neat YouTube datasets
based on the large-scale YouTube database provided by MineDojo. Specifically,
two rounds of filtering operations guarantee that the dataset covers enough
essential information and that the video-text pair is highly correlated.
Empirically, we demonstrate that the proposed method achieves better
performance on RL tasks compared with baselines. The code and datasets are
available at https://github.com/PKU-RL/CLIP4MC.
