  Odd-parity superconductivity is a rare and sought-for state of matter with a
potential for applications in topological quantum computing. Crystals with
staggered locally non-centrosymmetric structures have been proposed as
platforms where a magnetic field can induce a transition between even- and
odd-parity superconducting (SC) states. The superconductor CeRh$_2$As$_2$ with
a critical temperature $T_{\mathrm{c}}\approx0.4\,\mathrm{K}$ is likely the
first example material showing such a phase transition at a magnetic field
$\mu_{0}H^{*}=4\,\mathrm{T}$ applied along the crystallographic $c$ axis.
CeRh$_2$As$_2$ also undergoes a phase transition of an unknown origin at
$T_{0}=0.5\,\mathrm{K}$ and presents signs of an antiferromagnetism below
$0.25\,\mathrm{K}$. Under a hydrostatic pressure of
$P_0\approx0.5\,\mathrm{GPa}$, the $T_{0}$ order vanishes, resulting in a
quantum critical point. Here, using resistivity measurements under pressure, we
investigate how the correlations and normal-state orders affect the SC phase
switching. We find an enhancement of the in-plane critical field near $P_0$. At
the same time, the two SC states persist well past $P_{0}$, until at least
$2.7\,\mathrm{GPa}$ and $H^{*}$ is reduced to $0.3\,\mathrm{T}$, making the
putative odd-parity state stable almost down to zero field.
