  Asymmetry is a crucial characteristic of bilateral mammograms (Bi-MG) when
abnormalities are developing. It is widely utilized by radiologists for
diagnosis. The question of 'what the symmetrical Bi-MG would look like when the
asymmetrical abnormalities have been removed ?' has not yet received strong
attention in the development of algorithms on mammograms. Addressing this
question could provide valuable insights into mammographic anatomy and aid in
diagnostic interpretation. Hence, we propose a novel framework, DisAsymNet,
which utilizes asymmetrical abnormality transformer guided self-adversarial
learning for disentangling abnormalities and symmetric Bi-MG. At the same time,
our proposed method is partially guided by randomly synthesized abnormalities.
We conduct experiments on three public and one in-house dataset, and
demonstrate that our method outperforms existing methods in abnormality
classification, segmentation, and localization tasks. Additionally,
reconstructed normal mammograms can provide insights toward better
interpretable visual cues for clinical diagnosis. The code will be accessible
to the public.
