  In this paper, we consider the supervised pretrained transformer for a class
of sequential decision-making problems. The class of considered problems is a
subset of the general formulation of reinforcement learning in that there is no
transition probability matrix, and the class of problems covers bandits,
dynamic pricing, and newsvendor problems as special cases. Such a structure
enables the use of optimal actions/decisions in the pretraining phase, and the
usage also provides new insights for the training and generalization of the
pretrained transformer. We first note that the training of the transformer
model can be viewed as a performative prediction problem, and the existing
methods and theories largely ignore or cannot resolve the arisen
out-of-distribution issue. We propose a natural solution that includes the
transformer-generated action sequences in the training procedure, and it enjoys
better properties both numerically and theoretically. The availability of the
optimal actions in the considered tasks also allows us to analyze the
properties of the pretrained transformer as an algorithm and explains why it
may lack exploration and how this can be automatically resolved. Numerically,
we categorize the advantages of the pretrained transformer over the structured
algorithms such as UCB and Thompson sampling into three cases: (i) it better
utilizes the prior knowledge in the pretraining data; (ii) it can elegantly
handle the misspecification issue suffered by the structured algorithms; (iii)
for short time horizon such as $T\le50$, it behaves more greedy and enjoys much
better regret than the structured algorithms which are designed for asymptotic
optimality.
