  While large language models (LLMs) have been increasingly deployed across
tasks in language understanding and interactive decision-making, their
impressive performance is largely due to the comprehensive and in-depth domain
knowledge embedded within them. However, the extent of this knowledge can vary
across different domains. Existing methods often assume that LLMs already
possess such comprehensive and in-depth knowledge of their environment,
overlooking potential gaps in their understanding of actual world dynamics. To
address this gap, we introduce Discover, Verify, and Evolve (DiVE), a framework
that discovers world dynamics from a small number of demonstrations, verifies
the correctness of these dynamics, and evolves new, advanced dynamics tailored
to the current situation. Through extensive evaluations, we analyze the impact
of each component on performance and compare the automatically generated
dynamics from DiVE with human-annotated world dynamics. Our results demonstrate
that LLMs guided by DiVE can make better decisions, achieving rewards
comparable to human players in the Crafter environment.
