  Differentiable quantum architecture search (DQAS) is a gradient-based
framework to design quantum circuits automatically in the NISQ era. It was
motivated by such as low fidelity of quantum hardware, low flexibility of
circuit architecture, high circuit design cost, barren plateau (BP) problem,
and periodicity of weights. People used it to address error mitigation, unitary
decomposition, and quantum approximation optimization problems based on fixed
datasets. Quantum reinforcement learning (QRL) is a part of quantum machine
learning and often has various data. QRL usually uses a manually designed
circuit. However, the pre-defined circuit needs more flexibility for different
tasks, and the circuit design based on various datasets could become
intractable in the case of a large circuit. The problem of whether DQAS can be
applied to quantum deep Q-learning with various datasets is still open. The
main target of this work is to discover the capability of DQAS to solve quantum
deep Q-learning problems. We apply a gradient-based framework DQAS on
reinforcement learning tasks and evaluate it in two different environments -
cart pole and frozen lake. It contains input- and output weights, progressive
search, and other new features. The experiments conclude that DQAS can design
quantum circuits automatically and efficiently. The evaluation results show
significant outperformance compared to the manually designed circuit.
Furthermore, the performance of the automatically created circuit depends on
whether the super-circuit learned well during the training process. This work
is the first to show that gradient-based quantum architecture search is
applicable to QRL tasks.
