  Time series forecasting is crucial for applications across multiple domains
and various scenarios. Although Transformer models have dramatically shifted
the landscape of forecasting, their effectiveness remains debated. Recent
findings have indicated that simpler linear models might outperform complex
Transformer-based approaches, highlighting the potential for more streamlined
architectures. In this paper, we shift focus from the overall architecture of
the Transformer to the effectiveness of self-attentions for time series
forecasting. To this end, we introduce a new architecture, Cross-Attention-only
Time Series transformer (CATS), that rethinks the traditional Transformer
framework by eliminating self-attention and leveraging cross-attention
mechanisms instead. By establishing future horizon-dependent parameters as
queries and enhanced parameter sharing, our model not only improves long-term
forecasting accuracy but also reduces the number of parameters and memory
usage. Extensive experiment across various datasets demonstrates that our model
achieves superior performance with the lowest mean squared error and uses fewer
parameters compared to existing models.
