  Systematic reviews are crucial for evidence-based medicine as they
comprehensively analyse published research findings on specific questions.
Conducting such reviews is often resource- and time-intensive, especially in
the screening phase, where abstracts of publications are assessed for inclusion
in a review. This study investigates the effectiveness of using zero-shot large
language models~(LLMs) for automatic screening. We evaluate the effectiveness
of eight different LLMs and investigate a calibration technique that uses a
predefined recall threshold to determine whether a publication should be
included in a systematic review. Our comprehensive evaluation using five
standard test collections shows that instruction fine-tuning plays an important
role in screening, that calibration renders LLMs practical for achieving a
targeted recall, and that combining both with an ensemble of zero-shot models
saves significant screening time compared to state-of-the-art approaches.
