  Deep Learning has achieved tremendous results by pushing the frontier of
automation in diverse domains. Unfortunately, current neural network
architectures are not explainable by design. In this paper, we propose a novel
method that trains deep hypernetworks to generate explainable linear models.
Our models retain the accuracy of black-box deep networks while offering free
lunch explainability by design. Specifically, our explainable approach requires
the same runtime and memory resources as black-box deep models, ensuring
practical feasibility. Through extensive experiments, we demonstrate that our
explainable deep networks are as accurate as state-of-the-art classifiers on
tabular data. On the other hand, we showcase the interpretability of our method
on a recent benchmark by empirically comparing prediction explainers. The
experimental results reveal that our models are not only as accurate as their
black-box deep-learning counterparts but also as interpretable as
state-of-the-art explanation techniques.
