  We give a proof of the irrationality of the $p$-adic zeta-values $\zeta_p(k)$
for $p=2,3$ and $k=2,3$. Such results were recently obtained by F.Calegari as
an application of overconvergent $p$-adic modular forms. In this paper we
present an approach using classical continued fractions discovered by
Stieltjes. In addition we show irrationality of some other $p$-adic $L$-series
values, and values of the $p$-adic Hurwitz zeta-function.
