  Emotion recognition in smart eyewear devices is highly valuable but
challenging. One key limitation of previous works is that the
expression-related information like facial or eye images is considered as the
only emotional evidence. However, emotional status is not isolated; it is
tightly associated with people's visual perceptions, especially those
sentimental ones. However, little work has examined such associations to better
illustrate the cause of different emotions. In this paper, we study the
emotionship analysis problem in eyewear systems, an ambitious task that
requires not only classifying the user's emotions but also semantically
understanding the potential cause of such emotions. To this end, we devise
EMOShip, a deep-learning-based eyewear system that can automatically detect the
wearer's emotional status and simultaneously analyze its associations with
semantic-level visual perceptions. Experimental studies with 20 participants
demonstrate that, thanks to the emotionship awareness, EMOShip not only
achieves superior emotion recognition accuracy over existing methods (80.2% vs.
69.4%), but also provides a valuable understanding of the cause of emotions.
Pilot studies with 20 participants further motivate the potential use of
EMOShip to empower emotion-aware applications, such as emotionship
self-reflection and emotionship life-logging.
