  With coronary artery disease (CAD) persisting to be one of the leading causes
of death worldwide, interest in supporting physicians with algorithms to speed
up and improve diagnosis is high. In clinical practice, the severeness of CAD
is often assessed with a coronary CT angiography (CCTA) scan and manually
graded with the CAD-Reporting and Data System (CAD-RADS) score. The clinical
questions this score assesses are whether patients have CAD or not (rule-out)
and whether they have severe CAD or not (hold-out). In this work, we reach new
state-of-the-art performance for automatic CAD-RADS scoring. We propose using
severity-based label encoding, test time augmentation (TTA) and model
ensembling for a task-specific deep learning architecture. Furthermore, we
introduce a novel task- and model-specific, heuristic coronary segment
labeling, which subdivides coronary trees into consistent parts across
patients. It is fast, robust, and easy to implement. We were able to raise the
previously reported area under the receiver operating characteristic curve
(AUC) from 0.914 to 0.942 in the rule-out and from 0.921 to 0.950 in the
hold-out task respectively.
