  This paper addresses a new problem of weakly-supervised online action
segmentation in instructional videos. We present a framework to segment
streaming videos online at test time using Dynamic Programming and show its
advantages over greedy sliding window approach. We improve our framework by
introducing the Online-Offline Discrepancy Loss (OODL) to encourage the
segmentation results to have a higher temporal consistency. Furthermore, only
during training, we exploit frame-wise correspondence between multiple views as
supervision for training weakly-labeled instructional videos. In particular, we
investigate three different multi-view inference techniques to generate more
accurate frame-wise pseudo ground-truth with no additional annotation cost. We
present results and ablation studies on two benchmark multi-view datasets,
Breakfast and IKEA ASM. Experimental results show efficacy of the proposed
methods both qualitatively and quantitatively in two domains of cooking and
assembly.
