  Large language models (LLMs) have achieved impressive pro-gress on several
open-world tasks. Recently, using LLMs to build embodied agents has been a
hotspot. This paper proposes STEVE, a comprehensive and visionary embodied
agent in the Minecraft virtual environment. STEVE comprises three key
components: vision perception, language instruction, and code action. Vision
perception involves interpreting visual information in the environment, which
is then integrated into the LLMs component with agent state and task
instruction. Language instruction is responsible for iterative reasoning and
decomposing complex tasks into manageable guidelines. Code action generates
executable skill actions based on retrieval in skill database, enabling the
agent to interact effectively within the Minecraft environment. We also collect
STEVE-21K dataset, which includes 600+ vision-environment pairs, 20K knowledge
question-answering pairs, and 200+ skill-code pairs. We conduct continuous
block search, knowledge question and answering, and tech tree mastery to
evaluate the performance. Extensive experiments show that STEVE achieves at
most 1.5x faster unlocking key tech trees and 2.5x quicker in block search
tasks.
