  This paper presents a deep reinforcement learning approach for synthesizing
unitaries into quantum circuits. Unitary synthesis aims to identify a quantum
circuit that represents a given unitary while minimizing circuit depth, total
gate count, a specific gate count, or a combination of these factors. While
past research has focused predominantly on continuous gate sets, synthesizing
unitaries from the parameter-free Clifford+T gate set remains a challenge.
Although the time complexity of this task will inevitably remain exponential in
the number of qubits for general unitaries, reducing the runtime for simple
problem instances still poses a significant challenge. In this study, we apply
the tree-search method Gumbel AlphaZero to solve the problem for a subset of
exactly synthesizable Clifford+T unitaries. Our approach can synthesize
unitaries for up to five qubits generated from the set of randomized quantum
circuits with up to 60 gates. Furthermore, our inference times are around 30
seconds on a single GPU on average, surpassing state-of-the-art algorithms
QuantumCircuitOpt and MIN-T-SYNTH for higher qubit numbers. Our work provides a
competitive baseline for synthesis algorithms to be developed in the upcoming
years.
