  Machine learning models have been deployed in mobile networks to deal with
massive data from different layers to enable automated network management and
intelligence on devices. To overcome high communication cost and severe privacy
concerns of centralized machine learning, federated learning (FL) has been
proposed to achieve distributed machine learning among networked devices. While
the computation and communication limitation has been widely studied, the
impact of on-device storage on the performance of FL is still not explored.
Without an effective data selection policy to filter the massive streaming data
on devices, classical FL can suffer from much longer model training time
($4\times$) and significant inference accuracy reduction ($7\%$), observed in
our experiments. In this work, we take the first step to consider the online
data selection for FL with limited on-device storage. We first define a new
data valuation metric for data evaluation and selection in FL with theoretical
guarantees for speeding up model convergence and enhancing final model
accuracy, simultaneously. We further design {\ttfamily ODE}, a framework of
\textbf{O}nline \textbf{D}ata s\textbf{E}lection for FL, to coordinate
networked devices to store valuable data samples. Experimental results on one
industrial dataset and three public datasets show the remarkable advantages of
{\ttfamily ODE} over the state-of-the-art approaches. Particularly, on the
industrial dataset, {\ttfamily ODE} achieves as high as $2.5\times$ speedup of
training time and $6\%$ increase in inference accuracy, and is robust to
various factors in practical environments.
