  If a quantum system evolves in a noncyclic fashion the corresponding
geometric phase or holonomy may not be fully defined. Off-diagonal geometric
phases have been developed to deal with such cases. Here, we generalize these
phases to the non-Abelian case, by introducing off-diagonal holonomies that
involve evolution of more than one subspace of the underlying Hilbert space.
Physical realizations of the off-diagonal holonomies in adiabatic evolution and
interferometry are put forward.
