  This paper explores a Deep Reinforcement Learning (DRL) approach for
designing image-based control for edge robots to be implemented on Field
Programmable Gate Arrays (FPGAs). Although FPGAs are more power-efficient than
CPUs and GPUs, a typical DRL method cannot be applied since they are composed
of many Logic Blocks (LBs) for high-speed logical operations but low-speed
real-number operations. To cope with this problem, we propose a novel DRL
algorithm called Binarized P-Network (BPN), which learns image-input control
policies using Binarized Convolutional Neural Networks (BCNNs). To alleviate
the instability of reinforcement learning caused by a BCNN with low function
approximation accuracy, our BPN adopts a robust value update scheme called
Conservative Value Iteration, which is tolerant of function approximation
errors. We confirmed the BPN's effectiveness through applications to a visual
tracking task in simulation and real-robot experiments with FPGA.
