  We introduce a framework for the use of large language models (LLMs) in
Building Understandable Messaging for Policy and Evidence Review (BUMPER). LLMs
are proving capable of providing interfaces for understanding and synthesizing
large databases of diverse media. This presents an exciting opportunity to
supercharge the translation of scientific evidence into policy and action,
thereby improving livelihoods around the world. However, these models also pose
challenges related to access, trust-worthiness, and accountability. The BUMPER
framework is built atop a scientific knowledge base (e.g., documentation, code,
survey data) by the same scientists (e.g., individual contributor, lab,
consortium). We focus on a solution that builds trustworthiness through
transparency, scope-limiting, explicit-checks, and uncertainty measures. LLMs
are rapidly being adopted and consequences are poorly understood. The framework
addresses open questions regarding the reliability of LLMs and their use in
high-stakes applications. We provide a worked example in health policy for a
model designed to inform measles control programs. We argue that this framework
can facilitate accessibility of and confidence in scientific evidence for
policymakers, drive a focus on policy-relevance and translatability for
researchers, and ultimately increase and accelerate the impact of scientific
knowledge used for policy decisions.
