  Neural Information Retrieval (NIR) has significantly improved upon
heuristic-based IR systems. Yet, failures remain frequent, the models used
often being unable to retrieve documents relevant to the user's query. We
address this challenge by proposing a lightweight abstention mechanism tailored
for real-world constraints, with particular emphasis placed on the reranking
phase. We introduce a protocol for evaluating abstention strategies in a
black-box scenario, demonstrating their efficacy, and propose a simple yet
effective data-driven mechanism. We provide open-source code for experiment
replication and abstention implementation, fostering wider adoption and
application in diverse contexts.
