  All-optical spin switching (AOS) is a new phenomenon found in a small group
of magnetic media, where a single laser pulse can switch spins from one
direction to another, without assistance of a magnetic field, on a time scale
much shorter than existing magnetic technology. However, despite intensive
efforts over a decade, its underlying working principle remains elusive. Here
through manganese-based Heusler ferrimagnets, we show that a group of flat
bands around the Fermi level act as gateway states to form efficient channels
or spin switching, where their noncentrosymmetry allows us to correlate the
spin dynamics to the second-order optical response. To quantify their efficacy,
we introduce the third-rank Pancharatnam-Berry tensor (PB tensor),
$\boldsymbol{\eta}^{(3)}=\langle i |{\bf p} |m\rangle \langle m|{\bf p}
|f\rangle \langle f|{\bf p} |i\rangle,$ where $|i\rangle$, $|m\rangle$ and
$|f\rangle$ are initial, intermediate and final band states, respectively, and
${\bf p}$ is the momentum operator. A picture emerges: Those which show AOS,
such as the recently discovered Mn$_2$RuGa, always have a large PB tensor
element} but have a small sublattice spin moment ratio, consistent with the
prior experimental small remanence criterion. This does not only reveal that
the delicate balance between the large PB tensor element and the small
sublattice spin ratio plays a decisive role in AOS, but also, conceptually,
connects the $n$th-order nonlinear optics to $(n+1)$th-rank PB tensors in
general.
