  This paper presents an end-to-end model designed to improve automatic speech
recognition (ASR) for a particular speaker in a crowded, noisy environment. The
model utilizes a single-channel speech enhancement module that isolates the
speaker's voice from background noise (ConVoiFilter) and an ASR module. The
model can decrease ASR's word error rate (WER) from 80% to 26.4% through this
approach. Typically, these two components are adjusted independently due to
variations in data requirements. However, speech enhancement can create
anomalies that decrease ASR efficiency. By implementing a joint fine-tuning
strategy, the model can reduce the WER from 26.4% in separate tuning to 14.5%
in joint tuning. We openly share our pre-trained model to foster further
research hf.co/nguyenvulebinh/voice-filter.
