  We introduce scientific claim verification, a new task to select abstracts
from the research literature containing evidence that SUPPORTS or REFUTES a
given scientific claim, and to identify rationales justifying each decision. To
study this task, we construct SciFact, a dataset of 1.4K expert-written
scientific claims paired with evidence-containing abstracts annotated with
labels and rationales. We develop baseline models for SciFact, and demonstrate
that simple domain adaptation techniques substantially improve performance
compared to models trained on Wikipedia or political news. We show that our
system is able to verify claims related to COVID-19 by identifying evidence
from the CORD-19 corpus. Our experiments indicate that SciFact will provide a
challenging testbed for the development of new systems designed to retrieve and
reason over corpora containing specialized domain knowledge. Data and code for
this new task are publicly available at https://github.com/allenai/scifact. A
leaderboard and COVID-19 fact-checking demo are available at
https://scifact.apps.allenai.org.
