  The problem of search relevance in the E-commerce domain is a challenging one
since it involves understanding the intent of a user's short nuanced query and
matching it with the appropriate products in the catalog. This problem has
traditionally been addressed using language models (LMs) and graph neural
networks (GNNs) to capture semantic and inter-product behavior signals,
respectively. However, the rapid development of new architectures has created a
gap between research and the practical adoption of these techniques. Evaluating
the generalizability of these models for deployment requires extensive
experimentation on complex, real-world datasets, which can be non-trivial and
expensive. Furthermore, such models often operate on latent space
representations that are incomprehensible to humans, making it difficult to
evaluate and compare the effectiveness of different models. This lack of
interpretability hinders the development and adoption of new techniques in the
field. To bridge this gap, we propose Plug and Play Graph LAnguage Model
(PP-GLAM), an explainable ensemble of plug and play models. Our approach uses a
modular framework with uniform data processing pipelines. It employs additive
explanation metrics to independently decide whether to include (i) language
model candidates, (ii) GNN model candidates, and (iii) inter-product behavioral
signals. For the task of search relevance, we show that PP-GLAM outperforms
several state-of-the-art baselines as well as a proprietary model on real-world
multilingual, multi-regional e-commerce datasets. To promote better model
comprehensibility and adoption, we also provide an analysis of the
explainability and computational complexity of our model. We also provide the
public codebase and provide a deployment strategy for practical implementation.
