  Unsupervised graph representation learning has recently gained interest in
several application domains such as neuroscience, where modeling the diverse
morphology of cell types in the brain is one of the key challenges. It is
currently unknown how many excitatory cortical cell types exist and what their
defining morphological features are. Here we present GraphDINO, a purely
data-driven approach to learn low-dimensional representations of 3D neuronal
morphologies from unlabeled large-scale datasets. GraphDINO is a novel
transformer-based representation learning method for spatially-embedded graphs.
To enable self-supervised learning on transformers, we (1) developed data
augmentation strategies for spatially-embedded graphs, (2) adapted the
positional encoding and (3) introduced a novel attention mechanism,
AC-Attention, which combines attention-based global interaction between nodes
and classic graph convolutional processing. We show, in two different species
and across multiple brain areas, that this method yields morphological cell
type clusterings that are on par with manual feature-based classification by
experts, but without using prior knowledge about the structural features of
neurons. Moreover, it outperforms previous approaches on quantitative
benchmarks predicting expert labels. Our method could potentially enable
data-driven discovery of novel morphological features and cell types in
large-scale datasets. It is applicable beyond neuroscience in settings where
samples in a dataset are graphs and graph-level embeddings are desired.
