  Recent advances in artificial intelligence applied to biomedical text are
opening exciting opportunities for improving pharmacovigilance activities
currently burdened by the ever growing volumes of real world data. To fully
realize these opportunities, existing regulatory guidance and industry best
practices should be taken into consideration in order to increase the overall
trustworthiness of the system and enable broader adoption. In this paper we
present a case study on how to operationalize existing guidance for validated
AI systems in pharmacovigilance focusing on the specific task of medical
literature monitoring (MLM) of adverse events from the scientific literature.
We describe an AI system designed with the goal of reducing effort in MLM
activities built in close collaboration with subject matter experts and
considering guidance for validated systems in pharmacovigilance and AI
transparency. In particular we make use of public disclosures as a useful risk
control measure to mitigate system misuse and earn user trust. In addition we
present experimental results showing the system can significantly remove
screening effort while maintaining high levels of recall (filtering 55% of
irrelevant articles on average, for a target recall of 0.99 on suspected
adverse articles) and provide a robust method for tuning the desired recall to
suit a particular risk profile.
