  We present spectra and near-infrared images of a sample of faint radio
sources initially selected as promising high-redshift radio galaxy (HzRG)
candidates. We have determined redshifts for a total of 13 radio galaxies with
redshifts ranging from $0.52\le z \le 5.72$. Our sample probes radio
luminosities that are almost an order of magnitude fainter than previous large
samples at the highest redshifts. We use near-infrared photometry for a
subsample of these galaxies to calculate stellar masses using simple stellar
population models, and find stellar masses to be in the range $10^{10.8} -
10^{11.7} M_\odot$. We then compare our faint radio galaxies with brighter
radio galaxies at $z\ge2$ from the literature. We find that fainter radio
galaxies have lower Ly$\alpha$ luminosities and narrower full width at half
maxima compared to the bright ones, implying photoionisation by weaker AGN. We
also rule out the presence of strong shocks in faint HzRGs. The stellar masses
determined for faint HzRGs are lower than what has been observed for brighter
ones. We find that faint HzRG population in the redshift range $2-4$ forms a
bridge between star-forming and narrow-line AGN, whereas the ones at $z>4$ are
dominated by star-formation, and may be building up their stellar mass through
cold accretion of gas. Finally, we show that the overall redshift evolution of
radio sizes at $z>2$ is fully compatible with increased inverse Compton
scattering losses at high redshifts.
