  Automaton based approaches have enabled robots to perform various complex
tasks. However, most existing automaton based algorithms highly rely on the
manually customized representation of states for the considered task, limiting
its applicability in deep reinforcement learning algorithms. To address this
issue, by incorporating Transformer into reinforcement learning, we develop a
Double-Transformer-guided Temporal Logic framework (T2TL) that exploits the
structural feature of Transformer twice, i.e., first encoding the LTL
instruction via the Transformer module for efficient understanding of task
instructions during the training and then encoding the context variable via the
Transformer again for improved task performance. Particularly, the LTL
instruction is specified by co-safe LTL. As a semantics-preserving rewriting
operation, LTL progression is exploited to decompose the complex task into
learnable sub-goals, which not only converts non-Markovian reward decision
processes to Markovian ones, but also improves the sampling efficiency by
simultaneous learning of multiple sub-tasks. An environment-agnostic LTL
pre-training scheme is further incorporated to facilitate the learning of the
Transformer module resulting in an improved representation of LTL. The
simulation results demonstrate the effectiveness of the T2TL framework.
