  Differentially private (DP) stochastic convex optimization (SCO) is
ubiquitous in trustworthy machine learning algorithm design. This paper studies
the DP-SCO problem with streaming data sampled from a distribution and arrives
sequentially. We also consider the continual release model where parameters
related to private information are updated and released upon each new data,
often known as the online algorithms. Despite that numerous algorithms have
been developed to achieve the optimal excess risks in different $\ell_p$ norm
geometries, yet none of the existing ones can be adapted to the streaming and
continual release setting. To address such a challenge as the online convex
optimization with privacy protection, we propose a private variant of online
Frank-Wolfe algorithm with recursive gradients for variance reduction to update
and reveal the parameters upon each data. Combined with the adaptive
differential privacy analysis, our online algorithm achieves in linear time the
optimal excess risk when $1<p\leq 2$ and the state-of-the-art excess risk
meeting the non-private lower ones when $2<p\leq\infty$. Our algorithm can also
be extended to the case $p=1$ to achieve nearly dimension-independent excess
risk. While previous variance reduction results on recursive gradient have
theoretical guarantee only in the independent and identically distributed
sample setting, we establish such a guarantee in a non-stationary setting. To
demonstrate the virtues of our method, we design the first DP algorithm for
high-dimensional generalized linear bandits with logarithmic regret.
Comparative experiments with a variety of DP-SCO and DP-Bandit algorithms
exhibit the efficacy and utility of the proposed algorithms.
