  Objectives: To compare artificial intelligence (AI) as a second reader in
detecting lung nodules on chest X-rays (CXR) versus radiologists of two
binational institutions, and to evaluate AI performance when using two
different modes: automated versus assisted (additional remote radiologist
review).
  Methods: The CXR public database (n = 247) of the Japanese Society of
Radiological Technology with various types and sizes of lung nodules was
analyzed. Eight radiologists evaluated the CXR images with regard to the
presence of lung nodules and nodule conspicuity. After radiologist review, the
AI software processed and flagged the CXR with the highest probability of
missed nodules. The calculated accuracy metrics were the area under the curve
(AUC), sensitivity, specificity, F1 score, false negative case number (FN), and
the effect of different AI modes (automated/assisted) on the accuracy of nodule
detection.
  Results: For radiologists, the average AUC value was 0.77 $\pm$ 0.07, while
the average FN was 52.63 $\pm$ 17.53 (all studies) and 32 $\pm$ 11.59 (studies
containing a nodule of malignant etiology = 32% rate of missed malignant
nodules). Both AI modes -- automated and assisted -- produced an average
increase in sensitivity (by 14% and 12%) and of F1-score (5% and 6%) and a
decrease in specificity (by 10% and 3%, respectively).
  Conclusions: Both AI modes flagged the pulmonary nodules missed by
radiologists in a significant number of cases. AI as a second reader has a high
potential to improve diagnostic accuracy and radiology workflow. AI might
detect certain pulmonary nodules earlier than radiologists, with a potentially
significant impact on patient outcomes.
