  The extension of convolutional neural networks (CNNs) to non-Euclidean
geometries has led to multiple frameworks for studying manifolds. Many of those
methods have shown design limitations resulting in poor modelling of long-range
associations, as the generalisation of convolutions to irregular surfaces is
non-trivial. Motivated by the success of attention-modelling in computer
vision, we translate convolution-free vision transformer approaches to surface
data, to introduce a domain-agnostic architecture to study any surface data
projected onto a spherical manifold. Here, surface patching is achieved by
representing spherical data as a sequence of triangular patches, extracted from
a subdivided icosphere. A transformer model encodes the sequence of patches via
successive multi-head self-attention layers while preserving the sequence
resolution. We validate the performance of the proposed Surface Vision
Transformer (SiT) on the task of phenotype regression from cortical surface
metrics derived from the Developing Human Connectome Project (dHCP).
Experiments show that the SiT generally outperforms surface CNNs, while
performing comparably on registered and unregistered data. Analysis of
transformer attention maps offers strong potential to characterise subtle
cognitive developmental patterns.
