  The transformer models have been extensively used with good results in a wide
area of machine learning applications including Large Language Models and image
generation. Here, we inquire on the applicability of this approach to financial
time series. We first describe the dataset construction for two prototypical
situations: a mean reverting synthetic Ornstein-Uhlenbeck process on one hand
and real S&P500 data on the other hand. Then, we present in detail the proposed
Transformer architecture and finally we discuss some encouraging results. For
the synthetic data we predict rather accurately the next move, and for the
S&P500 we get some interesting results related to quadratic variation and
volatility prediction.
