  As the volume of long-form spoken-word content such as podcasts explodes,
many platforms desire to present short, meaningful, and logically coherent
segments extracted from the full content. Such segments can be consumed by
users to sample content before diving in, as well as used by the platform to
promote and recommend content. However, little published work is focused on the
segmentation of spoken-word content, where the errors (noise) in transcripts
generated by automatic speech recognition (ASR) services poses many challenges.
Here we build a novel dataset of complete transcriptions of over 400 podcast
episodes, in which we label the position of introductions in each episode.
These introductions contain information about the episodes' topics, hosts, and
guests, providing a valuable summary of the episode content, as it is created
by the authors. We further augment our dataset with word substitutions to
increase the amount of available training data. We train three Transformer
models based on the pre-trained BERT and different augmentation strategies,
which achieve significantly better performance compared with a static embedding
model, showing that it is possible to capture generalized, larger-scale
structural information from noisy, loosely-organized speech data. This is
further demonstrated through an analysis of the models' inner architecture. Our
methods and dataset can be used to facilitate future work on the
structure-based segmentation of spoken-word content.
