  We develop an algorithm which can learn from partially labeled and
unsegmented sequential data. Most sequential loss functions, such as
Connectionist Temporal Classification (CTC), break down when many labels are
missing. We address this problem with Star Temporal Classification (STC) which
uses a special star token to allow alignments which include all possible tokens
whenever a token could be missing. We express STC as the composition of
weighted finite-state transducers (WFSTs) and use GTN (a framework for
automatic differentiation with WFSTs) to compute gradients. We perform
extensive experiments on automatic speech recognition. These experiments show
that STC can recover most of the performance of supervised baseline when up to
70% of the labels are missing. We also perform experiments in handwriting
recognition to show that our method easily applies to other sequence
classification tasks.
