  This paper explores the novel deep learning Transformers architectures for
high-frequency Bitcoin-USDT log-return forecasting and compares them to the
traditional Long Short-Term Memory models. A hybrid Transformer model, called
\textbf{HFformer}, is then introduced for time series forecasting which
incorporates a Transformer encoder, linear decoder, spiking activations, and
quantile loss function, and does not use position encoding. Furthermore,
possible high-frequency trading strategies for use with the HFformer model are
discussed, including trade sizing, trading signal aggregation, and minimal
trading threshold. Ultimately, the performance of the HFformer and Long
Short-Term Memory models are assessed and results indicate that the HFformer
achieves a higher cumulative PnL than the LSTM when trading with multiple
signals during backtesting.
