  Self-supervised pretraining has been extensively studied in language and
vision domains, where a unified model can be easily adapted to various
downstream tasks by pretraining representations without explicit labels. When
it comes to sequential decision-making tasks, however, it is difficult to
properly design such a pretraining approach that can cope with both
high-dimensional perceptual information and the complexity of sequential
control over long interaction horizons. The challenge becomes combinatorially
more complex if we want to pretrain representations amenable to a large variety
of tasks. To tackle this problem, in this work, we formulate a general
pretraining-finetuning pipeline for sequential decision making, under which we
propose a generic pretraining framework \textit{Self-supervised Multi-task
pretrAining with contRol Transformer (SMART)}. By systematically investigating
pretraining regimes, we carefully design a Control Transformer (CT) coupled
with a novel control-centric pretraining objective in a self-supervised manner.
SMART encourages the representation to capture the common essential information
relevant to short-term control and long-term control, which is transferrable
across tasks. We show by extensive experiments in DeepMind Control Suite that
SMART significantly improves the learning efficiency among seen and unseen
downstream tasks and domains under different learning scenarios including
Imitation Learning (IL) and Reinforcement Learning (RL). Benefiting from the
proposed control-centric objective, SMART is resilient to distribution shift
between pretraining and finetuning, and even works well with low-quality
pretraining datasets that are randomly collected.
