  Artificial intelligence (AI) research is routinely criticized for its real
and potential impacts on society, and we lack adequate institutional responses
to this criticism and to the responsibility that it reflects. AI research often
falls outside the purview of existing feedback mechanisms such as the
Institutional Review Board (IRB), which are designed to evaluate harms to human
subjects rather than harms to human society. In response, we have developed the
Ethics and Society Review board (ESR), a feedback panel that works with
researchers to mitigate negative ethical and societal aspects of AI research.
The ESR's main insight is to serve as a requirement for funding: researchers
cannot receive grant funding from a major AI funding program at our university
until the researchers complete the ESR process for the proposal. In this
article, we describe the ESR as we have designed and run it over its first year
across 41 proposals. We analyze aggregate ESR feedback on these proposals,
finding that the panel most commonly identifies issues of harms to minority
groups, inclusion of diverse stakeholders in the research plan, dual use, and
representation in data. Surveys and interviews of researchers who interacted
with the ESR found that 58% felt that it had influenced the design of their
research project, 100% are willing to continue submitting future projects to
the ESR, and that they sought additional scaffolding for reasoning through
ethics and society issues.
