  Voice technology has become ubiquitous recently. However, the accuracy, and
hence experience, in different languages varies significantly, which makes the
technology not equally inclusive. The availability of data for different
languages is one of the key factors affecting accuracy, especially in training
of all-neural end-to-end automatic speech recognition systems.
  Cross-lingual knowledge transfer and iterative pseudo-labeling are two
techniques that have been shown to be successful for improving the accuracy of
ASR systems, in particular for low-resource languages, like Ukrainian.
  Our goal is to train an all-neural Transducer-based ASR system to replace a
DNN-HMM hybrid system with no manually annotated training data. We show that
the Transducer system trained using transcripts produced by the hybrid system
achieves 18% reduction in terms of word error rate. However, using a
combination of cross-lingual knowledge transfer from related languages and
iterative pseudo-labeling, we are able to achieve 35% reduction of the error
rate.
