  In [Ramanujan J. 52 (2020), 275-290], Romik considered the Taylor expansion
of Jacobi's theta function $\theta_3(q)$ at $q=e^{-\pi}$ and encoded it in an
integer sequence $(d(n))_{n\ge0}$ for which he provided a recursive procedure
to compute the terms of the sequence. He observed intriguing behaviour of
$d(n)$ modulo primes and prime powers. Here we prove (1) that $d(n)$ eventually
vanishes modulo any prime power $p^e$ with $p\equiv3$ (mod 4), (2) that $d(n)$
is eventually periodic modulo any prime power $p^e$ with $p\equiv1$ (mod 4),
and (3) that $d(n)$ is purely periodic modulo any 2-power $2^e$. Our results
also provide more detailed information on period length, respectively from when
on the sequence vanishes or becomes periodic. The corresponding bounds may not
be optimal though, as computer data suggest. Our approach shows that the above
congruence properties hold at a much finer, polynomial level.
