  We report detailed specific heat measurements on the recently discovered
magnetic superconductor RbEuFe$_4$As$_4$. We investigated the superconducting
transition at 37K and extract the phase boundary for in and out-of plane fields
resulting in an anisotropy ratio of 1.8. An unusual cusp-like feature in the
calorimetric data near 14.9K marks the onset of a magnetic phase. Studying the
effect of small fields along the crystallographic $c$ axis, we resolve a shift
in the cusp position moving to lower temperatures. For in-plane fields the cusp
rapidly disappears and a broad shoulder that shifts to higher temperatures. We
are able to reproduce our measured calorimetry data quantitatively by
Monte-Carlo simulations of an anisotropic easy-plane 2D Heisenberg model. We
can thus show that (i) the spins are preferably in plane, (ii) the cusp in
specific heat is due to a Berezinskii-Kosterlitz-Thouless (BKT) transition, and
(iii) the high-temperature hump in higher fields marks a crossover from a
paramagnetically disordered to an ordered state. The extracted phase and
crossover boundaries from experiment and simulations agree very well.
