  Real-time inference is a challenge of real-world reinforcement learning due
to temporal differences in time-varying environments: the system collects data
from the past, updates the decision model in the present, and deploys it in the
future. We tackle a common belief that continually updating the decision is
optimal to minimize the temporal gap. We propose forecasting an online
reinforcement learning framework and show that strategically pausing decision
updates yields better overall performance by effectively managing aleatoric
uncertainty. Theoretically, we compute an optimal ratio between policy update
and hold duration, and show that a non-zero policy hold duration provides a
sharper upper bound on the dynamic regret. Our experimental evaluations on
three different environments also reveal that a non-zero policy hold duration
yields higher rewards compared to continuous decision updates.
