  Traditional approaches in offline reinforcement learning aim to learn the
optimal policy that maximizes the cumulative reward, also known as return.
However, as applications broaden, it becomes increasingly crucial to train
agents that not only maximize the returns, but align the actual return with a
specified target return, giving control over the agent's performance. Decision
Transformer (DT) optimizes a policy that generates actions conditioned on the
target return through supervised learning and is equipped with a mechanism to
control the agent using the target return. However, the action generation is
hardly influenced by the target return because DT's self-attention allocates
scarce attention scores to the return tokens. In this paper, we propose
Return-Aligned Decision Transformer (RADT), designed to effectively align the
actual return with the target return. RADT utilizes features extracted by
paying attention solely to the return, enabling the action generation to
consistently depend on the target return. Extensive experiments show that RADT
reduces the discrepancies between the actual return and the target return of
DT-based methods.
