  Predicting outcomes, such as survival or metastasis for individual cancer
patients is a crucial component of precision oncology. Machine learning (ML)
offers a promising way to exploit rich multi-modal data, including clinical
information and imaging to learn predictors of disease trajectory and help
inform clinical decision making. In this paper, we present a novel graph-based
approach to incorporate imaging characteristics of existing cancer spread to
local lymph nodes (LNs) as well as their connectivity patterns in a prognostic
ML model. We trained an edge-gated Graph Convolutional Network (Gated-GCN) to
accurately predict the risk of distant metastasis (DM) by propagating
information across the LN graph with the aid of soft edge attention mechanism.
In a cohort of 1570 head and neck cancer patients, the Gated-GCN achieves AUROC
of 0.757 for 2-year DM classification and $C$-index of 0.725 for lifetime DM
risk prediction, outperforming current prognostic factors as well as previous
approaches based on aggregated LN features. We also explored the importance of
graph structure and individual lymph nodes through ablation experiments and
interpretability studies, highlighting the importance of considering individual
LN characteristics as well as the relationships between regions of cancer
spread.
