  Solitonic symmetry has been believed to follow the homotopy-group
classification of topological solitons. Here, we point out a more sophisticated
algebraic structure when solitons of different dimensions coexist in the
spectrum. We uncover this phenomenon in a concrete quantum field theory, the
$4$d $\mathbb{C}P^1$ model. This model has two kinds of solitonic excitations,
vortices and hopfions, which would follow two $U(1)$ solitonic symmetries
according to homotopy groups. Nevertheless, we demonstrate the nonexistence of
the hopfion $U(1)$ symmetry by evaluating the hopfion charge of vortex
operators. We clarify that what conserves hopfion numbers is a non-invertible
symmetry generated by 3d spin topological quantum field theories (TQFTs). Its
invertible part is just $\mathbb{Z}_2$, which we recognize as a spin bordism
invariant. Compared with the 3d $\mathbb{C}P^1$ model, our work suggests a
unified description of solitonic symmetries and couplings to topological
phases.
