  Reinforcement learning has been intensively investigated and developed in
artificial intelligence in the absence of training data, such as autonomous
driving vehicles, robot control, internet advertising, and elastic optical
networks. However, the computational cost of reinforcement learning with deep
neural networks is extremely high and reducing the learning cost is a
challenging issue. We propose a photonic on-line implementation of
reinforcement learning using optoelectronic delay-based reservoir computing,
both experimentally and numerically. In the proposed scheme, we accelerate
reinforcement learning at a rate of several megahertz because there is no
required learning process for the internal connection weights in reservoir
computing. We perform two benchmark tasks, CartPole-v0 and MountanCar-v0 tasks,
to evaluate the proposed scheme. Our results represent the first hardware
implementation of reinforcement learning based on photonic reservoir computing
and pave the way for fast and efficient reinforcement learning as a novel
photonic accelerator.
