  Quantum error-correcting codes (QECCs) can eliminate the negative effects of
quantum noise, the major obstacle to the execution of quantum algorithms.
However, realizing practical quantum error correction (QEC) requires resolving
many challenges to implement a high-performance real-time decoding system. Many
decoding algorithms have been proposed and optimized in the past few decades,
of which neural network (NNs) based solutions have drawn an increasing amount
of attention due to their high efficiency. Unfortunately, previous works on
neural decoders are still at an early stage and have only relatively simple
architectures, which makes them unsuitable for practical QEC. In this work, we
propose a scalable, fast, and programmable neural decoding system to meet the
requirements of FTQEC for rotated surface codes (RSC). Firstly, we propose a
hardware-efficient NN decoding algorithm with relatively low complexity and
high accuracy. Secondly, we develop a customized hardware decoder with
architectural optimizations to reduce latency. Thirdly, our proposed
programmable architecture boosts the scalability and flexibility of the decoder
by maximizing parallelism. Fourthly, we build an FPGA-based decoding system
with integrated control hardware for evaluation. Our $L=5$ ($L$ is the code
distance) decoder achieves an extremely low decoding latency of 197 ns, and the
$L=7$ configuration also requires only 1.136 $\mu$s, both taking $2L$ rounds of
syndrome measurements. The accuracy results of our system are close to minimum
weight perfect matching (MWPM). Furthermore, our programmable architecture
reduces hardware resource consumption by up to $3.0\times$ with only a small
latency loss. We validated our approach in real-world scenarios by conducting a
proof-of-concept benchmark with practical noise models, including one derived
from experimental data gathered from physical hardware.
