  Machine learning models with explainable predictions are increasingly sought
after, especially for real-world, mission-critical applications that require
bias detection and risk mitigation. Inherent interpretability, where a model is
designed from the ground-up for interpretability, provides intuitive insights
and transparent explanations on model prediction and performance. In this
paper, we present CoLabel, an approach to build interpretable models with
explanations rooted in the ground truth. We demonstrate CoLabel in a vehicle
feature extraction application in the context of vehicle make-model recognition
(VMMR). CoLabel performs VMMR with a composite of interpretable features such
as vehicle color, type, and make, all based on interpretable annotations of the
ground truth labels. First, CoLabel performs corroborative integration to join
multiple datasets that each have a subset of desired annotations of color,
type, and make. Then, CoLabel uses decomposable branches to extract
complementary features corresponding to desired annotations. Finally, CoLabel
fuses them together for final predictions. During feature fusion, CoLabel
harmonizes complementary branches so that VMMR features are compatible with
each other and can be projected to the same semantic space for classification.
With inherent interpretability, CoLabel achieves superior performance to the
state-of-the-art black-box models, with accuracy of 0.98, 0.95, and 0.94 on
CompCars, Cars196, and BoxCars116K, respectively. CoLabel provides intuitive
explanations due to constructive interpretability, and subsequently achieves
high accuracy and usability in mission-critical situations.
