  We report the discovery of CeLiBi$_2$, the first example of a material in the
tetragonal Ce$TX_2$ ($T$ = transition metal; $X$ = pnictogen) family wherein an
alkali cation replaces the typical transition metal. Magnetic susceptibility
and neutron powder diffraction measurements are consistent with a crystal-field
$\Gamma_6$ ground state Kramers doublet that orders antiferromagnetically below
$T_N = 3.4$ K with an incommensurate propagation wave vector ${\bf{k}} = (0,
0.0724(4), 0.5)$ that generates a nanometric modulation of the magnetic
structure. The best model of the ordered state is an elliptical cycloid with Ce
moments primarily residing in the $ab$ plane. This is highly unusual, as all
other $\Gamma_6$ Ce$TX_2$ members order ferromagnetically. Further, we observe
an atypical hard-axis metamagnetic transition at $2$ T in magnetostriction,
magnetization, and resistivity measurements. CeLiBi$_2$ is a rare example of a
highly conductive material with dominant skew scattering leading to a large
anomalous Hall effect. Quantum oscillations with five frequencies arise in
magnetostriction and magnetic susceptibility data to $T = 30$ K and $\mu_0H =
55$ T, which indicate small Fermi pockets of light carriers with effective
masses as low as 0.07$m_e$. Density functional theory calculations indicate
that square-net Dirac-like Bi$-p$ bands are responsible for these ultralight
carriers. Together, our results show that CeLiBi$_2$ enables multiple atypical
magnetic and electronic properties in a single clean material.
