  Learning policies from offline datasets through offline reinforcement
learning (RL) holds promise for scaling data-driven decision-making and
avoiding unsafe and costly online interactions. However, real-world data
collected from sensors or humans often contains noise and errors, posing a
significant challenge for existing offline RL methods. Our study indicates that
traditional offline RL methods based on temporal difference learning tend to
underperform Decision Transformer (DT) under data corruption, especially when
the amount of data is limited. This suggests the potential of sequential
modeling for tackling data corruption in offline RL. To further unleash the
potential of sequence modeling methods, we propose Robust Decision Transformer
(RDT) by incorporating several robust techniques. Specifically, we introduce
Gaussian weighted learning and iterative data correction to reduce the effect
of corrupted data. Additionally, we leverage embedding dropout to enhance the
model's resistance to erroneous inputs. Extensive experiments on MoJoCo,
KitChen, and Adroit tasks demonstrate RDT's superior performance under diverse
data corruption compared to previous methods. Moreover, RDT exhibits remarkable
robustness in a challenging setting that combines training-time data corruption
with testing-time observation perturbations. These results highlight the
potential of robust sequence modeling for learning from noisy or corrupted
offline datasets, thereby promoting the reliable application of offline RL in
real-world tasks.
