  Large language models (LLMs) have been successfully adapted for interactive
decision-making tasks like web navigation. While achieving decent performance,
previous methods implicitly assume a forward-only execution mode for the model,
where they only provide oracle trajectories as in-context examples to guide the
model on how to reason in the environment. Consequently, the model could not
handle more challenging scenarios not covered in the in-context examples, e.g.,
mistakes, leading to sub-optimal performance. To address this issue, we propose
to model the interactive task as state space exploration, where the LLM agent
transitions among a pre-defined set of states by performing actions to complete
the task. This formulation enables flexible backtracking, allowing the model to
recover from errors easily. We evaluate our proposed LLM Agent with State-Space
ExploRation (LASER) on both the WebShop task and amazon.com. Experimental
results show that LASER significantly outperforms previous methods and closes
the gap with human performance on the web navigation task.
