  To achieve near-zero training error in a classification problem, the layers
of a feed-forward network have to disentangle the manifolds of data points with
different labels, to facilitate the discrimination. However, excessive class
separation can bring to overfitting since good generalisation requires learning
invariant features, which involve some level of entanglement. We report on
numerical experiments showing how the optimisation dynamics finds
representations that balance these opposing tendencies with a non-monotonic
trend. After a fast segregation phase, a slower rearrangement (conserved across
data sets and architectures) increases the class entanglement.The training
error at the inversion is stable under subsampling, and across network
initialisations and optimisers, which characterises it as a property solely of
the data structure and (very weakly) of the architecture. The inversion is the
manifestation of tradeoffs elicited by well-defined and maximally stable
elements of the training set, coined ``stragglers'', particularly influential
for generalisation.
