  Preliminary attempts at incorporating the critical role of astrocytes - cells
that constitute more than 50% of human brain cells - in brain-inspired
neuromorphic computing remain in infancy. This paper seeks to delve deeper into
various key aspects of neuron-synapse-astrocyte interactions to mimic
self-attention mechanisms in Transformers. The cross-layer perspective explored
in this work involves bio-plausible modeling of Hebbian and pre-synaptic
plasticities in neuron-astrocyte networks, incorporating effects of
non-linearities and feedback along with algorithmic formulations to map the
neuron-astrocyte computations to self-attention mechanism and evaluating the
impact of incorporating bio-realistic effects from the machine learning
application side. Our analysis on sentiment and image classification tasks on
the IMDB and CIFAR10 datasets underscores the importance of constructing
Astromorphic Transformers from both accuracy and learning speed improvement
perspectives.
