  CeRh$_6$Ge$_4$ is an unusual example of a stoichiometric heavy fermion
ferromagnet, which can be cleanly tuned by hydrostatic pressure to a quantum
critical point. In order to understand the origin of this anomalous behavior,
we have characterized the magnetic ordering and crystalline electric field
(CEF) scheme of this system. While magnetic Bragg peaks are not resolved in
neutron powder diffraction, coherent oscillations are observed in zero-field
$\mu$SR below $T_{\rm C}$, which are consistent with in-plane ferromagnetic
ordering consisting of reduced Ce moments. From analyzing the magnetic
susceptibility and inelastic neutron scattering, we propose a CEF-level scheme
which accounts for the easy-plane magnetocrystalline anisotropy, where the low
lying first excited CEF exhibits significantly stronger hybridization than the
ground state. These results suggest that the orbital anisotropy of the ground
state and low lying excited state doublets are important for realizing
anisotropic electronic coupling between the $f$- and conduction electrons,
which gives rise to the highly anisotropic hybridization observed in
photoemission experiments.
