  This paper revisits the problem of sampling and transmitting status updates
through a channel with random delay under a sampling frequency constraint
\cite{sun_17_tit}. We use the Age of Information (AoI) to characterize the
status information freshness at the receiver. The goal is to design a sampling
policy that can minimize the average AoI when the statistics of delay is
unknown. We reformulate the problem as the optimization of a renewal-reward
process, and propose an online sampling strategy based on the Robbins-Monro
algorithm. We prove that the proposed algorithm satisfies the sampling
frequency constraint. Moreover, when the transmission delay is bounded and its
distribution is absolutely continuous, the average AoI obtained by the proposed
algorithm converges to the minimum AoI when the number of samples $K$ goes to
infinity with probability 1. We show that the optimality gap decays with rate
$\mathcal{O}\left(\ln K/K\right)$, and the proposed algorithm is minimax rate
optimal. Simulation results validate the performance of our proposed algorithm.
