  In this paper, we tackle the new Language-Based Audio Retrieval task proposed
in DCASE 2022. Firstly, we introduce a simple, scalable architecture which ties
both the audio and text encoder together. Secondly, we show that using this
architecture along with contrastive loss allows the model to significantly beat
the performance of the baseline model. Finally, in addition to having an
extremely low training memory requirement, we are able to use pretrained models
as it is without needing to finetune them. We test our methods and show that
using a combination of our methods beats the baseline scores significantly.
