  Magnetic excitations and magnetic structure of EuRbFe$_4$As$_4$ were
investigated by inelastic neutron scattering (INS), neutron diffraction, and
random phase approximation (RPA) calculations. Below the superconducting
transition temperature $T_\text{c}=36.5$~K, the INS spectra exhibit the neutron
spin resonances at $Q_\text{res}=1.27(2)$~$\text{\AA}^{-1}$ and
$1.79(3)$~$\text{\AA}^{-1}$. They correspond to the $\mathbf{Q}=(0.5,0.5,1)$
and $(0.5,0.5,3)$ nesting wave vectors, showing three dimensional nature of the
band structure. The characteristic energy of the neutron spin resonance is
$E_\text{res}=17.7(3)$~meV corresponding to $5.7(1)k_\text{B}T_\text{c}$.
Observation of the neutron spin resonance mode and our RPA calculations in
conjunction with the recent optical conductivity measurements are indicative of
the $s_\pm$ superconducting pairing symmetry in EuRbFe$_4$As$_4$. In addition
to the neutron spin resonance mode, upon decreasing temperature below the
magnetic transition temperature $T_\text{N}=15$~K, the spin wave excitation
originating in the long-range magnetic order of the Eu sublattice was observed
in the low-energy inelastic channel. Single-crystal neutron diffraction
measurements demonstrate that the magnetic propagation vector of the Eu
sublattice is $\mathbf{k}=(0, 0, 0.25)$, representing the three-dimensional
antiferromagnetic order. Linear spin wave calculations assuming the obtained
magnetic structure with the intra- and inter-plane nearest neighbor exchange
couplings of $J_1/k_\text{B}=-1.31$~K and $J_c/k_\text{B}=0.08$~K can reproduce
quantitatively the observed spin wave excitation. Our results show that
superconductivity and long-range magnetic order of Eu coexist in
EuRbFe$_4$As$_4$ whereas the coupling between them is rather weak.
