  The success of the multilingual automatic speech recognition systems
empowered many voice-driven applications. However, measuring the performance of
such systems remains a major challenge, due to its dependency on manually
transcribed speech data in both mono- and multilingual scenarios. In this
paper, we propose a novel multilingual framework -- eWER3 -- jointly trained on
acoustic and lexical representation to estimate word error rate. We demonstrate
the effectiveness of eWER3 to (i) predict WER without using any internal states
from the ASR and (ii) use the multilingual shared latent space to push the
performance of the close-related languages. We show our proposed multilingual
model outperforms the previous monolingual word error rate estimation method
(eWER2) by an absolute 9\% increase in Pearson correlation coefficient (PCC),
with better overall estimation between the predicted and reference WER.
