  Let $p$ be an odd prime, $N$ a square-free odd positive integer prime to $p$,
$\pi$ a $p$-ordinary cohomological irreducible cuspidal automorphic
representation of $\mathrm{GSp}_4(\mathbb{A}_\mathbb{Q})$ of principal level
$N$ and Iwahori level at $p$. Using a $p$-integral version of Rallis inner
product formula and modularity theorems for $\mathrm{GSp}_{4/\mathbb{Q}}$ and
$\mathrm{U}_{4/\mathbb{Q}}$, we establish an identity between the $p$-part of
the critical value at $1$ of the degree $5$ $L$-function of $\pi$ twisted by
the non-trivial quadratic Dirichlet character $\xi$ associated to the extension
$\mathbb{Q}(\sqrt{-N})/\mathbb{Q}$ and the $p$-part of the Selmer group of the
degree $5$ Galois representation associated to $\pi$ twisted by $\xi$, under
certain conditions on the residual Galois representation.
