  Tayler instability of toroidal magnetic fields $B_\phi$ is broadly invoked as
a trigger for turbulence and angular momentum transport in stars. This paper
presents a systematic revision of the linear stability analysis for a rotating,
magnetized, and stably stratified star. For plausible configurations of
$B_\phi$, instability requires diffusive processes: viscosity, magnetic
diffusivity, or thermal/compositional diffusion. Our results reveal a new
physical picture, demonstrating how different diffusive effects independently
trigger instability of two types of waves in the rotating star: magnetostrophic
waves and inertial waves. It develops via overstability of the waves, whose
growth rate sharply peaks at some characteristic wavenumbers. We determine
instability conditions for each wave branch and find the characteristic
wavenumbers. The results are qualitatively different for stars with magnetic
Prandtl number $Pm\ll 1$ (e.g. the Sun) and $Pm\gg 1$ (e.g. protoneutron
stars). The parameter dependence of unstable modes suggests a non-universal
scaling of the possible Tayler-Spruit dynamo.
