  Text-based game environments are challenging because agents must deal with
long sequences of text, execute compositional actions using text and learn from
sparse rewards. We address these challenges by proposing Language Decision
Transformers (LDTs), a framework that is based on transformer language models
and decision transformers (DTs). Our LDTs extend DTs with 3 components: (1)
exponential tilt to guide the agent towards high obtainable goals, (2) novel
goal conditioning methods yielding better results than the traditional
return-to-go (sum of all future rewards), and (3) a model of future
observations that improves agent performance. LDTs are the first to address
offline RL with DTs on these challenging games. Our experiments show that LDTs
achieve the highest scores among many different types of agents on some of the
most challenging Jericho games, such as Enchanter.
