  In this paper we address the problem of continuous fine-grained action
segmentation, in which multiple actions are present in an unsegmented video
stream. The challenge for this task lies in the need to represent the
hierarchical nature of the actions and to detect the transitions between
actions, allowing us to localise the actions within the video effectively. We
propose a novel recurrent semi-supervised Generative Adversarial Network (GAN)
model for continuous fine-grained human action segmentation. Temporal context
information is captured via a novel Gated Context Extractor (GCE) module,
composed of gated attention units, that directs the queued context information
through the generator model, for enhanced action segmentation. The GAN is made
to learn features in a semi-supervised manner, enabling the model to perform
action classification jointly with the standard, unsupervised, GAN learning
procedure. We perform extensive evaluations on different architectural variants
to demonstrate the importance of the proposed network architecture, and show
that it is capable of outperforming current state-of-the-art on three
challenging datasets: 50 Salads, MERL Shopping and Georgia Tech Egocentric
Activities dataset.
