  At temperatures well below the Fermi temperature $T_F$, the coupling of
magnetic fluctuations to particle-hole excitations in a two-component Fermi gas
makes the transition to itinerant ferromagnetism a first order phase
transition. This effect is not described by the paradigm of Landau's theory of
phase transitions, which assumes the free energy is an analytic function of the
order parameter and predicts a second order phase transition. On the other
hand, despite that larger symmetry often introduces larger degeneracies in the
low-lying states, here we show that for a Fermi gas with SU($N > 2$)-symmetry
in three space dimensions the ferromangetic phase transition is first order in
agreement with the predictions of Landau's theory [M. A. Cazalilla \emph{et
al}. New J. of Phys. {\bf 11} 103033 (2009)]. By performing unrestricted
Hartree-Fock calculations for an SU($N > 2$)-symmetric Fermi gas with short
range interactions, we find the order parameter undergoes a finite jump across
the transition. In addition, we do not observe any tri-critical point up to
temperatures $T \simeq 0.5\: T_F$, for which the thermal smearing of the Fermi
surface is subtantial. Going beyond mean-field, we find that the coupling of
magnetic fluctuations to particle-hole excitations makes the transition more
abrupt and further enhances the tendency of the gas to become fully polarized
for smaller values of $N$ and the gas parameter $k_F a_s$. In our study, we
also clarify the role of time reversal symmetry in the microscopic Hamiltonian
and obtain the temperature dependence of Tan's contact. For the latter, the
presence of the tri-critical point for $N = 2$ leads to a more pronounced
temperature dependence around the transition than for SU($N > 2$)-symmetric
gases.
