  There is currently a large gap in performance between the statistically
rigorous methods like linear regression or additive splines and the powerful
deep methods using neural networks. Previous works attempting to close this gap
have failed to fully investigate the exponentially growing number of feature
combinations which deep networks consider automatically during training. In
this work, we develop a tractable selection algorithm to efficiently identify
the necessary feature combinations by leveraging techniques in feature
interaction detection. Our proposed Sparse Interaction Additive Networks (SIAN)
construct a bridge from these simple and interpretable models to fully
connected neural networks. SIAN achieves competitive performance against
state-of-the-art methods across multiple large-scale tabular datasets and
consistently finds an optimal tradeoff between the modeling capacity of neural
networks and the generalizability of simpler methods.
