  Diffeological spaces firstly introduced by J.M. Souriau in the 1980s are a
natural generalization of smooth manifolds. However, optimization techniques
are only known on manifolds so far. Generalizing these techniques to
diffeological spaces is very challenging because of several reasons. One of the
main reasons is that there are various definitions of tangent spaces which do
not coincide. Additionally, one needs to deal with a generalization of a
Riemannian space in order to define gradients which are indispensable for
optimization methods. This paper is devoted to an optimization technique on
diffeological spaces. Thus, one main aim of this paper is a suitable definition
of a tangent space in view to optimization methods. Based on this definition,
we present a diffeological Riemannian space and a diffeological gradient, which
we need to formulate an optimization algorithm on diffeological spaces.
Moreover, in order to be able to update the iterates in an optimization
algorithm on diffeological spaces, we present a diffeological retraction and
the Levi-Civita connection on diffeological spaces. We give examples for the
novel objects and apply the presented diffeological algorithm to an
optimization problem.
