  Upon deployment to edge devices, it is often desirable for a model to further
learn from streaming data to improve accuracy. However, extracting
representative features from such data is challenging because it is typically
unlabeled, non-independent and identically distributed (non-i.i.d), and is seen
only once. To mitigate this issue, a common strategy is to maintain a small
data buffer on the edge device to hold the most representative data for further
learning. As most data is either never stored or quickly discarded, identifying
the most representative data to avoid significant information loss becomes
critical. In this paper, we propose an on-device framework that addresses this
issue by condensing incoming data into more informative samples. Specifically,
to effectively handle unlabeled incoming data, we propose a pseudo-labeling
technique designed for unlabeled on-device learning environments. Additionally,
we develop a dataset condensation technique that only requires little
computation resources. To counteract the effects of noisy labels during the
condensation process, we further utilize a contrastive learning objective to
improve the purity of class data within the buffer. Our empirical results
indicate substantial improvements over existing methods, particularly when
buffer capacity is severely restricted. For instance, with a buffer capacity of
just one sample per class, our method achieves an accuracy that outperforms the
best existing baseline by 58.4% on the CIFAR-10 dataset.
