  Breast cancer is one of the leading causes of mortality among women
worldwide. Early detection and risk assessment play a crucial role in improving
survival rates. Therefore, annual or biennial mammograms are often recommended
for screening in high-risk groups. Mammograms are typically interpreted by
expert radiologists based on the Breast Imaging Reporting and Data System
(BI-RADS), which provides a uniform way to describe findings and categorizes
them to indicate the level of concern for breast cancer. Recently, machine
learning (ML) and computational approaches have been developed to automate and
improve the interpretation of mammograms. However, both BI-RADS and the
ML-based methods focus on the analysis of data from the present and sometimes
the most recent prior visit. While it is clear that temporal changes in image
features of the longitudinal scans should carry value for quantifying breast
cancer risk, no prior work has conducted a systematic study of this. In this
paper, we extend a state-of-the-art ML model to ingest an arbitrary number of
longitudinal mammograms and predict future breast cancer risk. On a large-scale
dataset, we demonstrate that our model, LoMaR, achieves state-of-the-art
performance when presented with only the present mammogram. Furthermore, we use
LoMaR to characterize the predictive value of prior visits. Our results show
that longer histories (e.g., up to four prior annual mammograms) can
significantly boost the accuracy of predicting future breast cancer risk,
particularly beyond the short-term. Our code and model weights are available at
https://github.com/batuhankmkaraman/LoMaR.
