  We calculate the phase diagram of the SU($N$) Hubbard model describing
fermionic alkaline earth atoms in a square optical lattice with on-average one
atom per site, using a slave-rotor mean-field approximation. We find that the
chiral spin liquid predicted for $N\ge5$ and large interactions passes through
a fractionalized state with a spinon Fermi surface as interactions are
decreased before transitioning to a weakly interacting metal. We also show that
by adding an artificial uniform magnetic field with flux per plaquette
$2\pi/N$, the chiral spin liquid becomes the ground state for all $N\ge 3$ at
large interactions, persists to weaker interactions, and its spin gap
increases, suggesting that the spin liquid physics will persist to higher
temperatures. We discuss potential methods to realize the artificial gauge
fields and detect the predicted phases.
