  Transformers have demonstrated remarkable efficacy in forecasting time series
data. However, their extensive dependence on self-attention mechanisms demands
significant computational resources, thereby limiting their practical
applicability across diverse tasks, especially in multimodal problems. In this
work, we propose a new architecture, called perceiver-CDF, for modeling
cumulative distribution functions (CDF) of time series data. Our approach
combines the perceiver architecture with a copula-based attention mechanism
tailored for multimodal time series prediction. By leveraging the perceiver,
our model efficiently transforms high-dimensional and multimodal data into a
compact latent space, thereby significantly reducing computational demands.
Subsequently, we implement a copula-based attention mechanism to construct the
joint distribution of missing data for prediction. Further, we propose an
output variance testing mechanism to effectively mitigate error propagation
during prediction. To enhance efficiency and reduce complexity, we introduce
midpoint inference for the local attention mechanism. This enables the model to
efficiently capture dependencies within nearby imputed samples without
considering all previous samples. The experiments on the unimodal and
multimodal benchmarks consistently demonstrate a 20% improvement over
state-of-the-art methods while utilizing less than half of the computational
resources.
