  Human activity recognition (HAR) has been a classic research problem. In
particular, with recent machine learning (ML) techniques, the recognition task
has been largely investigated by companies and integrated into their products
for customers. However, most of them apply a predefined activity set and
conduct the learning process on the cloud, hindering specific personalizations
from end users (i.e., edge devices). Even though recent progress in Incremental
Learning allows learning new-class data on the fly, the learning process is
generally conducted on the cloud, requiring constant data exchange between
cloud and edge devices, thus leading to data privacy issues. In this paper, we
propose PILOTE, which pushes the incremental learning process to the extreme
edge, while providing reliable data privacy and practical utility, e.g., low
processing latency, personalization, etc. In particular, we consider the
practical challenge of extremely limited data during the incremental learning
process on edge, where catastrophic forgetting is required to be handled in a
practical way. We validate PILOTE with extensive experiments on human activity
data collected from mobile sensors. The results show PILOTE can work on edge
devices with extremely limited resources while providing reliable performance.
