  Quantum computing has the potential to solve problems that are intractable
for classical systems, yet the high error rates in contemporary quantum devices
often exceed tolerable limits for useful algorithm execution. Quantum Error
Correction (QEC) mitigates this by employing redundancy, distributing quantum
information across multiple data qubits and utilizing syndrome qubits to
monitor their states for errors. The syndromes are subsequently interpreted by
a decoding algorithm to identify and correct errors in the data qubits. This
task is complex due to the multiplicity of error sources affecting both data
and syndrome qubits as well as syndrome extraction operations. Additionally,
identical syndromes can emanate from different error sources, necessitating a
decoding algorithm that evaluates syndromes collectively. Although machine
learning (ML) decoders such as multi-layer perceptrons (MLPs) and convolutional
neural networks (CNNs) have been proposed, they often focus on local syndrome
regions and require retraining when adjusting for different code distances. We
introduce a transformer-based QEC decoder which employs self-attention to
achieve a global receptive field across all input syndromes. It incorporates a
mixed loss training approach, combining both local physical error and global
parity label losses. Moreover, the transformer architecture's inherent
adaptability to variable-length inputs allows for efficient transfer learning,
enabling the decoder to adapt to varying code distances without retraining.
  Evaluation on six code distances and ten different error configurations
demonstrates that our model consistently outperforms non-ML decoders, such as
Union Find (UF) and Minimum Weight Perfect Matching (MWPM), and other ML
decoders, thereby achieving best logical error rates. Moreover, the transfer
learning can save over 10x of training cost.
