  Cancer is a highly heterogeneous condition that can occur almost anywhere in
the human body. 18F-fluorodeoxyglucose is an imaging modality commonly used to
detect cancer due to its high sensitivity and clear visualisation of the
pattern of metabolic activity. Nonetheless, as cancer is highly heterogeneous,
it is challenging to train general-purpose discriminative cancer detection
models, with data availability and disease complexity often cited as a limiting
factor. Unsupervised anomaly detection models have been suggested as a putative
solution. These models learn a healthy representation of tissue and detect
cancer by predicting deviations from the healthy norm, which requires models
capable of accurately learning long-range interactions between organs and their
imaging patterns with high levels of expressivity. Such characteristics are
suitably satisfied by transformers, which have been shown to generate
state-of-the-art results in unsupervised anomaly detection by training on
normal data. This work expands upon such approaches by introducing multi-modal
conditioning of the transformer via cross-attention i.e. supplying anatomical
reference from paired CT. Using 294 whole-body PET/CT samples, we show that our
anomaly detection method is robust and capable of achieving accurate cancer
localization results even in cases where normal training data is unavailable.
In addition, we show the efficacy of this approach on out-of-sample data
showcasing the generalizability of this approach with limited training data.
Lastly, we propose to combine model uncertainty with a new kernel density
estimation approach, and show that it provides clinically and statistically
significant improvements when compared to the classic residual-based anomaly
maps. Overall, a superior performance is demonstrated against leading
state-of-the-art alternatives, drawing attention to the potential of these
approaches.
