  Fault diagnosis of rotating machinery is an important engineering problem. In
recent years, fault diagnosis methods based on the Convolutional Neural Network
(CNN) and Recurrent Neural Network (RNN) have been mature, but Transformer has
not been widely used in the field of fault diagnosis. To address these
deficiencies, a new method based on the Time Series Transformer (TST) is
proposed to recognize the fault mode of bearings. In this paper, our
contributions include: Firstly, we designed a tokens sequences generation
method which can handle data in 1D format, namely time series tokenizer. Then,
the TST combining time series tokenizer and Transformer was introduced.
Furthermore, the test results on the given dataset show that the proposed
method has better fault identification capability than the traditional CNN and
RNN models. Secondly, through the experiments, the effect of structural
hyperparameters such as subsequence length and embedding dimension on fault
diagnosis performance, computational complexity and parameters number of the
TST is analyzed in detail. The influence laws of some hyperparameters are
obtained. Finally, via t-Distributed Stochastic Neighbor Embedding (t-SNE)
dimensionality reduction method, the feature vectors in the embedding space are
visualized. On this basis, the working pattern of TST has been explained to a
certain extent. Moreover, by analyzing the distribution form of the feature
vectors, we find that compared with the traditional CNN and RNN models, the
feature vectors extracted by the method in this paper show the best intra-class
compactness and inter-class separability. These results further demonstrate the
effectiveness of the proposed method.
