  We propose a function-learning methodology with a control-theoretical
foundation. We parametrise the approximating function as the solution to a
control system on a reproducing-kernel Hilbert space, and propose several
methods to find the set of controls which bring the initial function as close
as possible to the target function. At first, we derive the expression for the
gradient of the cost function with respect to the controls that parametrise the
difference equations. This allows us to find the optimal controls by means of
gradient descent. In addition, we show how to compute derivatives of the
approximating functions with respect to the controls and describe two
optimisation methods relying on linear approximations of the approximating
functions. We show how the assumptions we make lead to results which are
coherent with Pontryagin's maximum principle. We test the optimisation methods
on two toy examples and on two higher-dimensional real-world problems, showing
that the approaches succeed in learning from real data and are versatile enough
to tackle learning tasks of different nature.
