  Ultrasonography is an important routine examination for breast cancer
diagnosis, due to its non-invasive, radiation-free and low-cost properties.
However, the diagnostic accuracy of breast cancer is still limited due to its
inherent limitations. It would be a tremendous success if we can precisely
diagnose breast cancer by breast ultrasound images (BUS). Many learning-based
computer-aided diagnostic methods have been proposed to achieve breast cancer
diagnosis/lesion classification. However, most of them require a pre-define ROI
and then classify the lesion inside the ROI. Conventional classification
backbones, such as VGG16 and ResNet50, can achieve promising classification
results with no ROI requirement. But these models lack interpretability, thus
restricting their use in clinical practice. In this study, we propose a novel
ROI-free model for breast cancer diagnosis in ultrasound images with
interpretable feature representations. We leverage the anatomical prior
knowledge that malignant and benign tumors have different spatial relationships
between different tissue layers, and propose a HoVer-Transformer to formulate
this prior knowledge. The proposed HoVer-Trans block extracts the inter- and
intra-layer spatial information horizontally and vertically. We conduct and
release an open dataset GDPH&SYSUCC for breast cancer diagnosis in BUS. The
proposed model is evaluated in three datasets by comparing with four CNN-based
models and two vision transformer models via five-fold cross validation. It
achieves state-of-the-art classification performance with the best model
interpretability. In the meanwhile, our proposed model outperforms two senior
sonographers on the breast cancer diagnosis when only one BUS image is given.
