  Many studies have examined the shortcomings of word error rate (WER) as an
evaluation metric for automatic speech recognition (ASR) systems. Since WER
considers only literal word-level correctness, new evaluation metrics based on
semantic similarity such as semantic distance (SD) and BERTScore have been
developed. However, we found that these metrics have their own limitations,
such as a tendency to overly prioritise keywords. We propose H_eval, a new
hybrid evaluation metric for ASR systems that considers both semantic
correctness and error rate and performs significantly well in scenarios where
WER and SD perform poorly. Due to lighter computation compared to BERTScore, it
offers 49 times reduction in metric computation time. Furthermore, we show that
H_eval correlates strongly with downstream NLP tasks. Also, to reduce the
metric calculation time, we built multiple fast and lightweight models using
distillation techniques
