  This paper makes several contributions to automatic lyrics transcription
(ALT) research. Our main contribution is a novel variant of the Multistreaming
Time-Delay Neural Network (MTDNN) architecture, called MSTRE-Net, which
processes the temporal information using multiple streams in parallel with
varying resolutions keeping the network more compact, and thus with a faster
inference and an improved recognition rate than having identical TDNN streams.
In addition, two novel preprocessing steps prior to training the acoustic model
are proposed. First, we suggest using recordings from both monophonic and
polyphonic domains during training the acoustic model. Second, we tag
monophonic and polyphonic recordings with distinct labels for discriminating
non-vocal silence and music instances during alignment. Moreover, we present a
new test set with a considerably larger size and a higher musical variability
compared to the existing datasets used in ALT literature, while maintaining the
gender balance of the singers. Our best performing model sets the
state-of-the-art in lyrics transcription by a large margin. For
reproducibility, we publicly share the identifiers to retrieve the data used in
this paper.
