  The quest for Kitaev quantum spin liquids has led to great interest in
honeycomb quantum magnets with strong spin-orbit coupling. It has been recently
proposed that even Mott insulators with $3d$ transition metal ions, having
nominally weak spin-orbit coupling, can realize such exotic physics. Motivated
by this, we study the rhombohedral honeycomb cobaltates CoTiO$_3$,
BaCo$_2$(PO$_4$)$_2$, and BaCo$_2$(AsO$_4$)$_2$, using $\textit{ab initio}$
density functional theory, which takes into account realistic crystal field
distortions and chemical information, in conjunction with exact diagonalization
numerics. We show that these Co$^{2+}$ magnets host $j_\text{eff}=1/2$ local
moments with highly anisotropic $g$-factors, and we extract their full spin
Hamiltonians including longer-range and anisotropic exchange couplings. For
CoTiO$_3$, we find a nearest-neighbor easy-plane ferromagnetic $XXZ$ model with
additional bond-dependent anisotropies and interlayer exchange, which supports
three-dimensional (3D) Dirac nodal line magnons. In contrast, for
BaCo$_2$(PO$_4$)$_2$ and BaCo$_2$(AsO$_4$)$_2$, we find a strongly suppressed
interlayer coupling, and significant frustration from additional third-neighbor
antiferromagnetic exchange mediated by P/As. Such bond-anisotropic $J_1$-$J_3$
spin models can support collinear zig-zag or coplanar spiral ground states; we
discuss their dynamical spin correlations which reveal a gapped Goldstone mode,
and argue that the effective parameters of these pseudospin-$1/2$ models may be
strongly renormalized by coupling to a low energy spin-exciton. Our results
call for re-examining proposals for realizing Kitaev spin liquids in the
honeycomb cobaltates.
