  We present the results of our investigation on nonlinear overstable rotating
magnetoconvection (RMC) in presence of vertical external magnetic field. We
focus on the dynamics appearing near the onset of convection by varying the
system control parameters, namely, the Taylor number ($\mathrm{Ta}$), the
Chandrasekhar number ($\mathrm{Q}$) and the Prandtl number ($\mathrm{Pr}$) in
the ranges $750\leq\mathrm{Ta}\leq10^6$, $0 < \mathrm{Q} \leq 10^3$ and $0 <
\mathrm{Pr} \leq 0.5$. Three dimensional (3D) direct numerical simulations
(DNS) of the governing equations and low-dimensional modeling of the system are
performed for this purpose. Extensive DNS in the specified parameter space
shows two qualitatively different onsets depending on $\mathrm{Ta}$,
$\mathrm{Q}$ and $\mathrm{Pr}$. In the first one, bistability appears at the
onset, where both subcritical and supercritical convection coexist, while only
supercritical convection is observed in the second one. Analysis of the
low-dimensional model reveals that a supercritical Hopf bifurcation is
responsible for the supercritical onset and a subcritical pitchfork bifurcation
is responsible for the subcritical onset. It is also observed that appearance
of subcritical convection at the onset has strong dependence on all three
control parameters $\mathrm{Ta}$, $\mathrm{Q}$ and $\mathrm{Pr}$. The scenario
of subcritical convection is found to disappear as $\mathrm{Pr}$ is increased
for fixed $\mathrm{Ta}$ and $\mathrm{Q}$. However, most striking findings of
the investigation is that the increment in $\mathrm{Ta}$ for fixed $\mathrm{Q}$
and $\mathrm{Pr}$ opposes the subcritical convection, whereas the increment in
$\mathrm{Q}$ for fixed $\mathrm{Ta}$ and $\mathrm{Pr}$ favors it. This is in
sharp contrast with the earlier results reported in RMC.
