  Recent work has shown that Large Language Models (LLMs) can be incredibly
effective for offline reinforcement learning (RL) by representing the
traditional RL problem as a sequence modelling problem (Chen et al., 2021;
Janner et al., 2021). However many of these methods only optimize for high
returns, and may not extract much information from a diverse dataset of
trajectories. Generalized Decision Transformers (GDTs) (Furuta et al., 2021)
have shown that utilizing future trajectory information, in the form of
information statistics, can help extract more information from offline
trajectory data. Building upon this, we propose Skill Decision Transformer
(Skill DT). Skill DT draws inspiration from hindsight relabelling (Andrychowicz
et al., 2017) and skill discovery methods to discover a diverse set of
primitive behaviors, or skills. We show that Skill DT can not only perform
offline state-marginal matching (SMM), but can discovery descriptive behaviors
that can be easily sampled. Furthermore, we show that through purely
reward-free optimization, Skill DT is still competitive with supervised offline
RL approaches on the D4RL benchmark. The code and videos can be found on our
project page: https://github.com/shyamsn97/skill-dt
