  Pre-trained large transformer models have achieved remarkable performance in
the fields of natural language processing and computer vision. However, the
limited availability of public electroencephalogram (EEG) data presents a
unique challenge for extending the success of these models to EEG-based tasks.
To address this gap, we propose AdaCT, plug-and-play Adapters designed for
Converting Time series data into spatio-temporal 2D pseudo-images or text
forms. Essentially, AdaCT-I transforms multi-channel or lengthy single-channel
time series data into spatio-temporal 2D pseudo-images for fine-tuning
pre-trained vision transformers, while AdaCT-T converts short single-channel
data into text for fine-tuning pre-trained language transformers. The proposed
approach allows for seamless integration of pre-trained vision models and
language models in time series decoding tasks, particularly in EEG data
analysis. Experimental results on diverse benchmark datasets, including
Epileptic Seizure Recognition, Sleep-EDF, and UCI HAR, demonstrate the
superiority of AdaCT over baseline methods. Overall, we provide a promising
transfer learning framework for leveraging the capabilities of pre-trained
vision and language models in EEG-based tasks, thereby advancing the field of
time series decoding and enhancing interpretability in EEG data analysis. Our
code will be available at https://github.com/wangbxj1234/AdaCE.
