  Brain simulation, as one of the latest advances in artificial intelligence,
facilitates better understanding about how information is represented and
processed in the brain. The extreme complexity of human brain makes brain
simulations only feasible upon high-performance computing platforms.
Supercomputers with a large number of interconnected graphical processing units
(GPUs) are currently employed for supporting brain simulations. Therefore,
high-throughput low-latency inter-GPU communications in supercomputers play a
crucial role in meeting the performance requirements of brain simulation as a
highly time-sensitive application. In this paper, we first provide an overview
of the current parallelizing technologies for brain simulations using multi-GPU
architectures. Then, we analyze the challenges to communications for brain
simulation and summarize guidelines for communication design to address such
challenges. Furthermore, we propose a partitioning algorithm and a two-level
routing method to achieve efficient low-latency communications in multi-GPU
architecture for brain simulation. We report experiment results obtained on a
supercomputer with 2,000 GPUs for simulating a brain model with 10 billion
neurons to show that our approach can significantly improve communication
performance. We also discuss open issues and identify some research directions
for low-latency communication design for brain simulations.
