  Efficient and accurate product relevance assessment is critical for user
experiences and business success. Training a proficient relevance assessment
model requires high-quality query-product pairs, often obtained through
negative sampling strategies. Unfortunately, current methods introduce pooling
bias by mistakenly sampling false negatives, diminishing performance and
business impact. To address this, we present Bias-mitigating Hard Negative
Sampling (BHNS), a novel negative sampling strategy tailored to identify and
adjust for false negatives, building upon our original False Negative
Estimation algorithm. Our experiments in the Instacart search setting confirm
BHNS as effective for practical e-commerce use. Furthermore, comparative
analyses on public dataset showcase its domain-agnostic potential for diverse
applications.
