  In action understanding in indoor, we have to recognize human pose and action
considering privacy. Although camera images can be used for highly accurate
human action recognition, camera images do not preserve privacy. Therefore, we
propose a new task for human instance segmentation from invisible information,
especially airborne ultrasound, for action recognition. To perform instance
segmentation from invisible information, we first convert sound waves to
reflected sound directional images (sound images). Although the sound images
can roughly identify the location of a person, the detailed shape is ambiguous.
To address this problem, we propose a collaborative learning variational
autoencoder (CL-VAE) that simultaneously uses sound and RGB images during
training. In inference, it is possible to obtain instance segmentation results
only from sound images. As a result of performance verification, CL-VAE could
estimate human instance segmentations more accurately than conventional
variational autoencoder and some other models. Since this method can obtain
human segmentations individually, it could be applied to human action
recognition tasks with privacy protection.
