  The recent rapid development of language models (LMs) has attracted attention
in the field of time series, including multimodal time series modeling.
However, we note that current time series multimodal methods are biased, often
assigning a primary role to one modality while the other assumes a secondary
role. They overlook the mutual benefits and complementary of different
modalities. For example, in seizure diagnosis, relying solely on textual
clinical reports makes it difficult to pinpoint the area and type of the
disease, while electroencephalograms (EEGs) alone cannot provide an accurate
diagnosis without considering the symptoms. In this study, based on the
complementary information mining of time series multimodal data, we propose
DualTime, a Dual-adapter multimodal language model for Time series
representation implementing temporal-primary and textual-primary modeling
simultaneously. By injecting lightweight adaption tokens, the LM pipeline
shared by dual adapters encourages embedding alignment and achieves efficient
fine-tuning. Empirically, our method outperforms state-of-the-art models in
both supervised and unsupervised settings, highlighting the complementary
benefits of different modalities. In addition, we conduct few-shot label
transfer experiments, which further verifies the transferability and
expressiveness of our proposed DualTime.
