  Automatic Speech Recognition (ASR) systems predominantly cater to monolingual
inputs and struggle with the complexity introduced by mixed language audio. In
this paper, we present a novel Multi-Agent Data Generation Framework (MADGF) to
address this challenge. We finetune the open-source multilingual ASR model,
Whisper, utilizing our generated Mixed Cantonese and English (MCE) audio
dataset, Which achieved an impressive Mix Error Rate (MER) of 14.28%, 35.13%
lower than the original model. Meanwhile, single language recognition ability
is not affected, 12.6% Character Error Rate (CER) in Common voice zh-HK, 14.8%
Word Error Rate (WER) in Common voice en. However, these metrics do not
encompass all aspects critical to the ASR systems. Hence, we propose a novel
evaluation metric called Fidelity to the Original Audio, Accuracy, and Latency
(FAL).
