  Time series prediction plays a crucial role in various industrial fields. In
recent years, neural networks with a transformer backbone have achieved
remarkable success in many domains, including computer vision and NLP. In time
series analysis domain, some studies have suggested that even the simplest MLP
networks outperform advanced transformer-based networks on time series forecast
tasks. However, we believe these findings indicate there to be low-rank
properties in time series sequences. In this paper, we consider the low-pass
characteristics of transformers and try to incorporate the advantages of MLP.
We adopt skip-layer connections inspired by Unet into traditional transformer
backbone, thus preserving high-frequency context from input to output, namely
U-shaped Transformer. We introduce patch merge and split operation to extract
features with different scales and use larger datasets to fully make use of the
transformer backbone. Our experiments demonstrate that the model performs at an
advanced level across multiple datasets with relatively low cost.
