  With the rise of bidirectional encoder representations from Transformer
models in natural language processing, the speech community has adopted some of
their development methodologies. Therefore, the Wav2Vec models were introduced
to reduce the data required to obtain state-of-the-art results. This work
leverages this knowledge and improves the performance of the pre-trained speech
models by simply replacing the fine-tuning dense layer with a lateral
inhibition layer inspired by the biological process. Our experiments on
Romanian, a low-resource language, show an average improvement of 12.5% word
error rate (WER) using the lateral inhibition layer. In addition, we obtain
state-of-the-art results on both the Romanian Speech Corpus and the Robin
Technical Acquisition Corpus with 1.78% WER and 29.64% WER, respectively.
