  Single Index Models (SIMs) are simple yet flexible semi-parametric models for
machine learning, where the response variable is modeled as a monotonic
function of a linear combination of features. Estimation in this context
requires learning both the feature weights and the nonlinear function that
relates features to observations. While methods have been described to learn
SIMs in the low dimensional regime, a method that can efficiently learn SIMs in
high dimensions, and under general structural assumptions, has not been
forthcoming. In this paper, we propose computationally efficient algorithms for
SIM inference in high dimensions with structural constraints. Our general
approach specializes to sparsity, group sparsity, and low-rank assumptions
among others. Experiments show that the proposed method enjoys superior
predictive performance when compared to generalized linear models, and achieves
results comparable to or better than single layer feedforward neural networks
with significantly less computational cost.
