  Multi-label activity recognition is designed for recognizing multiple
activities that are performed simultaneously or sequentially in each video.
Most recent activity recognition networks focus on single-activities, that
assume only one activity in each video. These networks extract shared features
for all the activities, which are not designed for multi-label activities. We
introduce an approach to multi-label activity recognition that extracts
independent feature descriptors for each activity and learns activity
correlations. This structure can be trained end-to-end and plugged into any
existing network structures for video classification. Our method outperformed
state-of-the-art approaches on four multi-label activity recognition datasets.
To better understand the activity-specific features that the system generated,
we visualized these activity-specific features in the Charades dataset.
