  NeurIPS 2020 requested that research paper submissions include impact
statements on "potential nefarious uses and the consequences of failure."
However, as researchers, practitioners and system designers, a key challenge to
anticipating risks is overcoming what Clarke (1962) called 'failures of
imagination.' The growing research on bias, fairness, and transparency in
computational systems aims to illuminate and mitigate harms, and could thus
help inform reflections on possible negative impacts of particular pieces of
technical work. The prevalent notion of computational harms -- narrowly
construed as either allocational or representational harms -- does not fully
capture the open, context dependent, and unobservable nature of harms across
the wide range of AI infused systems.The current literature focuses on a small
range of examples of harms to motivate algorithmic fixes, overlooking the wider
scope of probable harms and the way these harms might affect different
stakeholders. The system affordances may also exacerbate harms in unpredictable
ways, as they determine stakeholders' control(including of non-users) over how
they use and interact with a system output. To effectively assist in
anticipating harmful uses, we argue that frameworks of harms must be
context-aware and consider a wider range of potential stakeholders, system
affordances, as well as viable proxies for assessing harms in the widest sense.
