  The vast majority of research on explainability focuses on
post-explainability rather than explainable modeling. Namely, an explanation
model is derived to explain a complex black box model built with the sole
purpose of achieving the highest performance possible. In part, this trend
might be driven by the misconception that there is a trade-off between
explainability and accuracy. Furthermore, the consequential work on Shapely
values, grounded in game theory, has also contributed to a new wave of
post-explainability research on better approximations for various machine
learning models, including deep learning models. We propose a new architecture
that inherently produces explainable predictions in the form of additive
feature attributions. Our approach learns a graph representation for each
record in the dataset. Attribute centric features are then derived from the
graph and fed into a contribution deep set model to produce the final
predictions. We show that our explainable model attains the same level of
performance as black box models. Finally, we provide an augmented model
training approach that leverages the missingness property and yields high
levels of consistency (as required for the Shapely values) without loss of
accuracy.
