  Recent advancements in language models have demonstrated remarkable
improvements in various natural language processing (NLP) tasks such as web
navigation. Supervised learning (SL) approaches have achieved impressive
performance while utilizing significantly less training data compared to
previous methods. However, these SL-based models fall short when compared to
reinforcement learning (RL) approaches, which have shown superior results. In
this paper, we propose a novel approach that combines SL and RL techniques over
the MiniWoB benchmark to leverage the strengths of both methods. We also
address a critical limitation in previous models' understanding of HTML
content, revealing a tendency to memorize target elements rather than
comprehend the underlying structure. To rectify this, we propose methods to
enhance true understanding and present a new baseline of results. Our
experiments demonstrate that our approach outperforms previous SL methods on
certain tasks using less data and narrows the performance gap with RL models,
achieving 43.58\% average accuracy in SL and 36.69\% when combined with a
multimodal RL approach. This study sets a new direction for future web
navigation and offers insights into the limitations and potential of language
modeling for computer tasks.
