  We present a study of the upper critical field, H$_{c2}$, of pristine and
proton-irradiated RbEuFe$_4$As$_4$ crystals in pulsed magnetic fields of up to
65 T. The data for H$_{c2}$ reveal pronounced downwards curvature, particularly
for the in-plane field orientation, and a superconducting anisotropy that
decreases with decreasing temperature. These features are indicative of Pauli
paramagnetic limiting. For the interpretation of these data, we use a model of
a clean single-band superconductor with an open Fermi surface in the shape of a
warped cylinder, which includes strong paramagnetic limiting. Fits to the data
reveal that the in-plane upper critical field is Pauli paramagnetic limited,
while the out-of-plane upper critical field is orbitally limited and that the
orbital and paramagnetic fields have opposite anisotropies. A consequence of
this particular combination is the unusual inversion of the anisotropy,
$H_{c2}^{ab} < H_{c2}^c$, of the irradiated sample at temperatures below 10 K.
The fits also yield an in-plane Maki parameter, $\alpha_M^{110} \approx$ 2.6,
exceeding the critical value for the formation of the
Fulde-Ferrell-Larkin-Ovchinnikov state. Nevertheless, the current measurements
did not reveal direct evidence for the occurrence of this state.
