  Inter-neuron communication delays are ubiquitous in physically realized
neural networks such as biological neural circuits and neuromorphic hardware.
These delays have significant and often disruptive consequences on network
dynamics during training and inference. It is therefore essential that
communication delays be accounted for, both in computational models of
biological neural networks and in large-scale neuromorphic systems.
Nonetheless, communication delays have yet to be comprehensively addressed in
either domain. In this paper, we first show that delays prevent
state-of-the-art continuous-time neural networks called Latent Equilibrium (LE)
networks from learning even simple tasks despite significant
overparameterization. We then propose to compensate for communication delays by
predicting future signals based on currently available ones. This conceptually
straightforward approach, which we call prospective messaging (PM), uses only
neuron-local information, and is flexible in terms of memory and computation
requirements. We demonstrate that incorporating PM into delayed LE networks
prevents reaction lags, and facilitates successful learning on Fourier
synthesis and autoregressive video prediction tasks.
