  Purpose: To investigate chest radiograph (CXR) classification performance of
vision transformers (ViT) and interpretability of attention-based saliency
using the example of pneumothorax classification.
  Materials and Methods: In this retrospective study, ViTs were fine-tuned for
lung disease classification using four public data sets: CheXpert, Chest X-Ray
14, MIMIC CXR, and VinBigData. Saliency maps were generated using transformer
multimodal explainability and gradient-weighted class activation mapping
(GradCAM). Classification performance was evaluated on the Chest X-Ray 14,
VinBigData, and SIIM-ACR data sets using the area under the receiver operating
characteristic curve analysis (AUC) and compared with convolutional neural
networks (CNNs). The explainability methods were evaluated with
positive/negative perturbation, sensitivity-n, effective heat ratio,
intra-architecture repeatability and interarchitecture reproducibility. In the
user study, three radiologists classified 160 CXRs with/without saliency maps
for pneumothorax and rated their usefulness.
  Results: ViTs had comparable CXR classification AUCs compared with
state-of-the-art CNNs 0.95 (95% CI: 0.943, 0.950) versus 0.83 (95%, CI 0.826,
0.842) on Chest X-Ray 14, 0.84 (95% CI: 0.769, 0.912) versus 0.83 (95% CI:
0.760, 0.895) on VinBigData, and 0.85 (95% CI: 0.847, 0.861) versus 0.87 (95%
CI: 0.868, 0.882) on SIIM ACR. Both saliency map methods unveiled a strong bias
toward pneumothorax tubes in the models. Radiologists found 47% of the
attention-based saliency maps useful and 39% of GradCAM. The attention-based
methods outperformed GradCAM on all metrics.
  Conclusion: ViTs performed similarly to CNNs in CXR classification, and their
attention-based saliency maps were more useful to radiologists and outperformed
GradCAM.
