  Sparse high-dimensional functions have arisen as a rich framework to study
the behavior of gradient-descent methods using shallow neural networks,
showcasing their ability to perform feature learning beyond linear models.
Amongst those functions, the simplest are single-index models $f(x) = \phi( x
\cdot \theta^*)$, where the labels are generated by an arbitrary non-linear
scalar link function $\phi$ applied to an unknown one-dimensional projection
$\theta^*$ of the input data. By focusing on Gaussian data, several recent
works have built a remarkable picture, where the so-called information exponent
(related to the regularity of the link function) controls the required sample
complexity. In essence, these tools exploit the stability and spherical
symmetry of Gaussian distributions. In this work, building from the framework
of \cite{arous2020online}, we explore extensions of this picture beyond the
Gaussian setting, where both stability or symmetry might be violated. Focusing
on the planted setting where $\phi$ is known, our main results establish that
Stochastic Gradient Descent can efficiently recover the unknown direction
$\theta^*$ in the high-dimensional regime, under assumptions that extend
previous works \cite{yehudai2020learning,wu2022learning}.
