  In recent years, self-supervised learning methods have shown significant
improvement for pre-training with unlabeled data and have proven helpful for
electrocardiogram signals. However, most previous pre-training methods for
electrocardiogram focused on capturing only global contextual representations.
This inhibits the models from learning fruitful representation of
electrocardiogram, which results in poor performance on downstream tasks.
Additionally, they cannot fine-tune the model with an arbitrary set of
electrocardiogram leads unless the models were pre-trained on the same set of
leads. In this work, we propose an ECG pre-training method that learns both
local and global contextual representations for better generalizability and
performance on downstream tasks. In addition, we propose random lead masking as
an ECG-specific augmentation method to make our proposed model robust to an
arbitrary set of leads. Experimental results on two downstream tasks, cardiac
arrhythmia classification and patient identification, show that our proposed
approach outperforms other state-of-the-art methods.
