  Recently, R. Tauraso established finite $p$-analogues of famous Ap\'ery
series for $\zeta(2)$ and $\zeta(3).$ In this paper, we present several
congruences for finite central binomial sums arising from the truncation of
Ap\'ery-type series for $\zeta(4)$ and $\zeta(5).$ We also prove a $p$-analogue
of Zeilberger's series for $\zeta(2)$ confirming a conjecture of Z. W. Sun.
