  Language-agnostic many-to-one end-to-end speech translation models can
convert audio signals from different source languages into text in a target
language. These models do not need source language identification, which
improves user experience. In some cases, the input language can be given or
estimated. Our goal is to use this additional language information while
preserving the quality of the other languages. We accomplish this by
introducing a simple and effective linear input network. The linear input
network is initialized as an identity matrix, which ensures that the model can
perform as well as, or better than, the original model. Experimental results
show that the proposed method can successfully enhance the specified language,
while keeping the language-agnostic ability of the many-to-one ST models.
