  Understanding the irregular electrical activity of atrial fibrillation (AFib)
has been a key challenge in electrocardiography. For serious cases of AFib,
catheter ablations are performed to collect intracardiac electrograms (EGMs).
EGMs offer intricately detailed and localized electrical activity of the heart
and are an ideal modality for interpretable cardiac studies. Recent
advancements in artificial intelligence (AI) has allowed some works to utilize
deep learning frameworks to interpret EGMs during AFib. Additionally, language
models (LMs) have shown exceptional performance in being able to generalize to
unseen domains, especially in healthcare. In this study, we are the first to
leverage pretrained LMs for finetuning of EGM interpolation and AFib
classification via masked language modeling. We formulate the EGM as a textual
sequence and present competitive performances on AFib classification compared
against other representations. Lastly, we provide a comprehensive
interpretability study to provide a multi-perspective intuition of the model's
behavior, which could greatly benefit the clinical use.
