  We introduce a general framework of stochastic online convex optimization to
obtain fast-rate stochastic regret bounds. We prove that algorithms such as
online newton steps and a scale-free 10 version of Bernstein online aggregation
achieve best-known rates in unbounded stochastic settings. We apply our
approach to calibrate parametric probabilistic forecasters of non-stationary
sub-gaussian time series. Our fast-rate stochastic regret bounds are any-time
valid. Our proofs combine self-bounded and Poissonnian inequalities for
martingales and sub-gaussian random variables, respectively, under a stochastic
exp-concavity assumption.
