  We study the stability of rotating scalar boson stars, comparing those made
from a simple massive complex scalar (referred to as mini boson stars), to
those with several different types of nonlinear interactions. To that end, we
numerically evolve the nonlinear Einstein-Klein-Gordon equations in 3D,
beginning with stationary boson star solutions. We show that the linear,
non-axisymmetric instability found in mini boson stars with azimuthal number
$m=1$ persists across the entire parameter space for these stars, though the
timescale diverges in the Newtonian limit. Therefore, any boson star with $m=1$
that is sufficiently far into the non-relativistic regime, where the leading
order mass term dominates, will be unstable, independent of the nonlinear
scalar self-interactions. However, we do find regions of $m=1$ boson star
parameter space where adding nonlinear interactions to the scalar potential
quenches the non-axisymmetric instability, both on the non-relativistic, and
the relativistic branches of solutions. We also consider select boson stars
with $m=2$, finding instability in all cases. For the cases exhibiting
instability, we follow the nonlinear development, finding a range of dynamics
including fragmentation into multiple unbound non-rotating stars, and formation
of binary black holes. Finally, we comment on the relationship between
stability and criteria based on the rotating boson star's frequency in relation
to that of a spherical boson star or the existence of a co-rotation point. The
boson stars that we find not to exhibit instability when evolved for many
dynamical times include rapidly rotating cases where the compactness is
comparable to that of a black hole or neutron star.
