  Supervised learning in deep neural networks is commonly performed using error
backpropagation. However, the sequential propagation of errors during the
backward pass limits its scalability and applicability to low-powered
neuromorphic hardware. Therefore, there is growing interest in finding local
alternatives to backpropagation. Recently proposed methods based on
forward-mode automatic differentiation suffer from high variance in large deep
neural networks, which affects convergence. In this paper, we propose the
Forward Direct Feedback Alignment algorithm that combines Activity-Perturbed
Forward Gradients with Direct Feedback Alignment and momentum. We provide both
theoretical proofs and empirical evidence that our proposed method achieves
lower variance than forward gradient techniques. In this way, our approach
enables faster convergence and better performance when compared to other local
alternatives to backpropagation and opens a new perspective for the development
of online learning algorithms compatible with neuromorphic systems.
