  In a previous work, we proposed a geometric framework to study a deep neural
network, seen as sequence of maps between manifolds, employing singular
Riemannian geometry. In this paper, we present an application of this
framework, proposing a way to build the class of equivalence of an input point:
such class is defined as the set of the points on the input manifold mapped to
the same output by the neural network. In other words, we build the preimage of
a point in the output manifold in the input space. In particular. we focus for
simplicity on the case of neural networks maps from n-dimensional real spaces
to (n - 1)-dimensional real spaces, we propose an algorithm allowing to build
the set of points lying on the same class of equivalence. This approach leads
to two main applications: the generation of new synthetic data and it may
provides some insights on how a classifier can be confused by small
perturbation on the input data (e.g. a penguin image classified as an image
containing a chihuahua). In addition, for neural networks from 2D to 1D real
spaces, we also discuss how to find the preimages of closed intervals of the
real line. We also present some numerical experiments with several neural
networks trained to perform non-linear regression tasks, including the case of
a binary classifier.
