  The relationship between brain connections and non-imaging phenotypes is
increasingly studied using deep neural networks. However, the local and global
properties of the brain's white matter networks are often overlooked in
convolutional network design. We introduce TractGraphFormer, a hybrid Graph
CNN-Transformer deep learning framework tailored for diffusion MRI
tractography. This model leverages local anatomical characteristics and global
feature dependencies of white matter structures. The Graph CNN module captures
white matter geometry and grey matter connectivity to aggregate local features
from anatomically similar white matter connections, while the Transformer
module uses self-attention to enhance global information learning.
Additionally, TractGraphFormer includes an attention module for interpreting
predictive white matter connections. In sex prediction tests, TractGraphFormer
shows strong performance in large datasets of children (n=9345) and young
adults (n=1065). Overall, our approach suggests that widespread connections in
the WM are predictive of the sex of an individual, and consistent predictive
anatomical tracts are identified across the two datasets. The proposed approach
highlights the potential of integrating local anatomical information and global
feature dependencies to improve prediction performance in machine learning with
diffusion MRI tractography.
