  Embedding-based neural retrieval is a prevalent approach to address the
semantic gap problem which often arises in product search on tail queries. In
contrast, popular queries typically lack context and have a broad intent where
additional context from users historical interaction can be helpful. In this
paper, we share our novel approach to address both: the semantic gap problem
followed by an end to end trained model for personalized semantic retrieval. We
propose learning a unified embedding model incorporating graph, transformer and
term-based embeddings end to end and share our design choices for optimal
tradeoff between performance and efficiency. We share our learnings in feature
engineering, hard negative sampling strategy, and application of transformer
model, including a novel pre-training strategy and other tricks for improving
search relevance and deploying such a model at industry scale. Our personalized
retrieval model significantly improves the overall search experience, as
measured by a 5.58% increase in search purchase rate and a 2.63% increase in
site-wide conversion rate, aggregated across multiple A/B tests - on live
traffic.
