  Egocentric visual context detection can support intelligence augmentation
applications. We created a wearable system, called PAL, for wearable,
personalized, and privacy-preserving egocentric visual context detection. PAL
has a wearable device with a camera, heart-rate sensor, on-device deep
learning, and audio input/output. PAL also has a mobile/web application for
personalized context labeling. We used on-device deep learning models for
generic object and face detection, low-shot custom face and context recognition
(e.g., activities like brushing teeth), and custom context clustering (e.g.,
indoor locations). The models had over 80\% accuracy in in-the-wild contexts
(~1000 images) and we tested PAL for intelligence augmentation applications
like behavior change. We have made PAL is open-source to further support
intelligence augmentation using personalized and privacy-preserving egocentric
visual contexts.
