  When a quantum state traverses a path, while being under the influence of a
gauge potential, it acquires a geometric phase that is often more than just a
scalar quantity. The variety of unitary transformations that can be realised by
this form of parallel transport depends crucially on the number of particles
involved in the evolution. Here, we introduce a particle-number threshold (PNT)
that assesses a system's capabilities to perform purely geometric manipulations
of quantum states. This threshold gives the minimal number of particles
necessary to fully exploit a system's potential to generate non-Abelian
geometric phases. Therefore, the PNT might be useful for evaluating the
resource demands of a holonomic quantum computer. We benchmark our findings on
bosonic systems relevant to linear and nonlinear quantum optics.
