  This work addresses the synthesis of optimal feedback control laws via
machine learning. In particular, the Averaged Feedback Learning Scheme (AFLS)
and a data driven method are considered. Hypotheses for each method ensuring
the convergence of the evaluation of the objective function of the underlying
control problem at the obtained feedback-laws towards the optimal value
function are provided. These hypotheses are connected to the regularity of the
value function and the stability of the dynamics. In the case of AFLS these
hypotheses only require H\"older continuity of the value function, whereas for
the data driven method the value function must be at least $C^2$. It is
demonstrated that these methods are connected via their optimality conditions.
Additionally, numerical experiments are provided by applying both methods to a
family control problems, parameterized by a positive real number which controls
the regularity of the value function. For small parameters the value function
is smooth and in contrast for large parameters it is non-differentiable, but
semi-concave. The results of the experiments indicate that both methods have a
similar performance for the case that the value function is smooth. On the
other hand, if the value function is not differentiable, AFLS has a better
performance which is consistent with the obtained convergence results.
