  There has been a growing interest in developing learner models to enhance
learning and teaching experiences in educational environments. However,
existing works have primarily focused on structured environments relying on
meticulously crafted representations of tasks, thereby limiting the agent's
ability to generalize skills across tasks. In this paper, we aim to enhance the
generalization capabilities of agents in open-ended text-based learning
environments by integrating Reinforcement Learning (RL) with Large Language
Models (LLMs). We investigate three types of agents: (i) RL-based agents that
utilize natural language for state and action representations to find the best
interaction strategy, (ii) LLM-based agents that leverage the model's general
knowledge and reasoning through prompting, and (iii) hybrid LLM-assisted RL
agents that combine these two strategies to improve agents' performance and
generalization. To support the development and evaluation of these agents, we
introduce PharmaSimText, a novel benchmark derived from the PharmaSim virtual
pharmacy environment designed for practicing diagnostic conversations. Our
results show that RL-based agents excel in task completion but lack in asking
quality diagnostic questions. In contrast, LLM-based agents perform better in
asking diagnostic questions but fall short of completing the task. Finally,
hybrid LLM-assisted RL agents enable us to overcome these limitations,
highlighting the potential of combining RL and LLMs to develop high-performing
agents for open-ended learning environments.
