  Understanding spatial location and relationships is a fundamental capability
for modern artificial intelligence systems. Insights from human spatial
cognition provide valuable guidance in this domain. Recent neuroscientific
discoveries have highlighted the role of grid cells as a fundamental neural
component for spatial representation, including distance computation, path
integration, and scale discernment. In this paper, we introduce a novel
positional encoding scheme inspired by Fourier analysis and the latest findings
in computational neuroscience regarding grid cells. Assuming that grid cells
encode spatial position through a summation of Fourier basis functions, we
demonstrate the translational invariance of the grid representation during
inner product calculations. Additionally, we derive an optimal grid scale ratio
for multi-dimensional Euclidean spaces based on principles of biological
efficiency. Utilizing these computational principles, we have developed a
**Grid**-cell inspired **Positional Encoding** technique, termed **GridPE**,
for encoding locations within high-dimensional spaces. We integrated GridPE
into the Pyramid Vision Transformer architecture. Our theoretical analysis
shows that GridPE provides a unifying framework for positional encoding in
arbitrary high-dimensional spaces. Experimental results demonstrate that GridPE
significantly enhances the performance of transformers, underscoring the
importance of incorporating neuroscientific insights into the design of
artificial intelligence systems.
