  We consider the fundamental problem of learning a single neuron $x
\mapsto\sigma(w^\top x)$ using standard gradient methods. As opposed to
previous works, which considered specific (and not always realistic) input
distributions and activation functions $\sigma(\cdot)$, we ask whether a more
general result is attainable, under milder assumptions. On the one hand, we
show that some assumptions on the distribution and the activation function are
necessary. On the other hand, we prove positive guarantees under mild
assumptions, which go beyond those studied in the literature so far. We also
point out and study the challenges in further strengthening and generalizing
our results.
