  Azure Cognitive Search (ACS) has emerged as a major contender in "Search as a
Service" cloud products in recent years. However, one of the major challenges
for ACS users is to improve the relevance of the search results for their
specific usecases. In this paper, we propose a novel method to find the optimal
ACS configuration that maximizes search relevance for a specific usecase
(product search, document search...) The proposed solution improves key online
marketplace metrics such as click through rates (CTR) by formulating the search
relevance problem as hyperparameter tuning. We have observed significant
improvements in real-world search call to action (CTA) rate in multiple
marketplaces by introducing optimized weights generated from the proposed
approach.
