  Egocentric videos offer fine-grained information for high-fidelity modeling
of human behaviors. Hands and interacting objects are one crucial aspect of
understanding a viewer's behaviors and intentions. We provide a labeled dataset
consisting of 11,243 egocentric images with per-pixel segmentation labels of
hands and objects being interacted with during a diverse array of daily
activities. Our dataset is the first to label detailed hand-object contact
boundaries. We introduce a context-aware compositional data augmentation
technique to adapt to out-of-distribution YouTube egocentric video. We show
that our robust hand-object segmentation model and dataset can serve as a
foundational tool to boost or enable several downstream vision applications,
including hand state classification, video activity recognition, 3D mesh
reconstruction of hand-object interactions, and video inpainting of hand-object
foregrounds in egocentric videos. Dataset and code are available at:
https://github.com/owenzlz/EgoHOS
