  Embedding based retrieval has seen its usage in a variety of search
applications like e-commerce, social networking search etc. While the approach
has demonstrated its efficacy in tasks like semantic matching and contextual
search, it is plagued by the problem of uncontrollable relevance. In this
paper, we conduct an analysis of embedding-based retrieval launched in early
2021 on our social network search engine, and define two main categories of
failures introduced by it, integrity and junkiness. The former refers to issues
such as hate speech and offensive content that can severely harm user
experience, while the latter includes irrelevant results like fuzzy text
matching or language mismatches. Efficient methods during model inference are
further proposed to resolve the issue, including indexing treatments and
targeted user cohort treatments, etc. Though being simple, we show the methods
have good offline NDCG and online A/B tests metrics gain in practice. We
analyze the reasons for the improvements, pointing out that our methods are
only preliminary attempts to this important but challenging problem. We put
forward potential future directions to explore.
