  Understanding the learning process of artificial neural networks requires
clarifying the structure of the parameter space within which learning takes
place. A neural network parameter's functional equivalence class is the set of
parameters implementing the same input--output function. For many
architectures, almost all parameters have a simple and well-documented
functional equivalence class. However, there is also a vanishing minority of
reducible parameters, with richer functional equivalence classes caused by
redundancies among the network's units.
  In this paper, we give an algorithmic characterisation of unit redundancies
and reducible functional equivalence classes for a single-hidden-layer
hyperbolic tangent architecture. We show that such functional equivalence
classes are piecewise-linear path-connected sets, and that for parameters with
a majority of redundant units, the sets have a diameter of at most 7 linear
segments.
