  Traditional visual navigation methods of micro aerial vehicle (MAV) usually
calculate a passable path that satisfies the constraints depending on a prior
map. However, these methods have issues such as high demand for computing
resources and poor robustness in face of unfamiliar environments. Aiming to
solve the above problems, we propose a neuromorphic reinforcement learning
method (Neuro-Planner) that combines spiking neural network (SNN) and deep
reinforcement learning (DRL) to realize MAV 3D visual navigation with depth
camera. Specifically, we design spiking actor network based on two-state LIF
(TS-LIF) neurons and its encoding-decoding schemes for efficient inference.
Then our improved hybrid deep deterministic policy gradient (HDDPG) and
TS-LIF-based spatio-temporal back propagation (STBP) algorithms are used as the
training framework for actor-critic network architecture. To verify the
effectiveness of the proposed Neuro-Planner, we carry out detailed comparison
experiments with various SNN training algorithm (STBP, BPTT and SLAYER) in the
software-in-the-loop (SITL) simulation framework. The navigation success rate
of our HDDPG-STBP is 4.3\% and 5.3\% higher than that of the original DDPG in
the two evaluation environments. To the best of our knowledge, this is the
first work combining neuromorphic computing and deep reinforcement learning for
MAV 3D visual navigation task.
