  Velocity dispersion ($\sigma$) is a key driver for galaxy structure and
evolution. We here present a comprehensive semi-empirical approach to compute
$\sigma$ via detailed Jeans modelling assuming both a constant and
scale-dependent mass-to-light ratio $M^*/L$. We compare with a large sample of
local galaxies from MaNGA and find that both models can reproduce the
Faber-Jackson (FJ) relation and the weak dependence of $\sigma$ on
bulge-to-total ratio $B/T$ (for $B/T\gtrsim 0.25$). The dynamical-to-stellar
mass ratio within $R\lesssim R_e$ can be fully accounted for by a gradient in
$M^*/L$. We then build velocity dispersion evolutionary tracks
$\sigma_{ap}[M^*,z]$ (within an aperture) along the main progenitor dark matter
haloes assigning stellar masses, effective radii and Sersic indices via a
variety of abundance matching and empirically motivated relations. We find: 1)
clear evidence for downsizing in $\sigma_{ap}[M^*,z]$ along the progenitor
tracks; 2) at fixed stellar mass $\sigma\propto(1+z)^{0.2-0.3}$ depending on
the presence or not of a gradient in $M^*/L$. We extract $\sigma_{ap}[M^*,z]$
from the TNG50 hydrodynamic simulation and find very similar results to our
models with constant $M^*/L$. The increasing dark matter fraction within $R_e$
tends to flatten the $\sigma_{ap}[M^*,z]$ along the progenitors at $z \gtrsim
1$ in constant $M^*/L$ models, while $\sigma_{ap}[M^*,z]$ have a steeper
evolution in the presence of a stellar gradient. We then show that a
combination of mergers and gas accretion are likely responsible for the
constant or increasing $\sigma_{ap}[M^*,z]$ with time. Finally, our
$\sigma_{ap}[M^*,z]$ are consistent with a nearly constant and steep
$M_{bh}-\sigma$ relation at $z\lesssim 2$, with black hole masses derived from
the $L_X-M^*$ relation.
