  A growing body of literature has proposed formal approaches to audit
algorithmic systems for biased and harmful behaviors. While formal auditing
approaches have been greatly impactful, they often suffer major blindspots,
with critical issues surfacing only in the context of everyday use once systems
are deployed. Recent years have seen many cases in which everyday users of
algorithmic systems detect and raise awareness about harmful behaviors that
they encounter in the course of their everyday interactions with these systems.
However, to date little academic attention has been granted to these bottom-up,
user-driven auditing processes. In this paper, we propose and explore the
concept of everyday algorithm auditing, a process in which users detect,
understand, and interrogate problematic machine behaviors via their day-to-day
interactions with algorithmic systems. We argue that everyday users are
powerful in surfacing problematic machine behaviors that may elude detection
via more centrally-organized forms of auditing, regardless of users' knowledge
about the underlying algorithms. We analyze several real-world cases of
everyday algorithm auditing, drawing lessons from these cases for the design of
future platforms and tools that facilitate such auditing behaviors. Finally, we
discuss work that lies ahead, toward bridging the gaps between formal auditing
approaches and the organic auditing behaviors that emerge in everyday use of
algorithmic systems.
