  This paper describes the ON-TRAC Consortium translation systems developed for
two challenge tracks featured in the Evaluation Campaign of IWSLT 2022:
low-resource and dialect speech translation. For the Tunisian Arabic-English
dataset (low-resource and dialect tracks), we build an end-to-end model as our
joint primary submission, and compare it against cascaded models that leverage
a large fine-tuned wav2vec 2.0 model for ASR. Our results show that in our
settings pipeline approaches are still very competitive, and that with the use
of transfer learning, they can outperform end-to-end models for speech
translation (ST). For the Tamasheq-French dataset (low-resource track) our
primary submission leverages intermediate representations from a wav2vec 2.0
model trained on 234 hours of Tamasheq audio, while our contrastive model uses
a French phonetic transcription of the Tamasheq audio as input in a Conformer
speech translation architecture jointly trained on automatic speech
recognition, ST and machine translation losses. Our results highlight that
self-supervised models trained on smaller sets of target data are more
effective to low-resource end-to-end ST fine-tuning, compared to large
off-the-shelf models. Results also illustrate that even approximate phonetic
transcriptions can improve ST scores.
