  Algorithms in public services such as child welfare, criminal justice, and
education are increasingly being used to make high-stakes decisions about human
lives. Drawing upon findings from a two-year ethnography conducted at a child
welfare agency, we highlight how algorithmic systems are embedded within a
complex decision-making ecosystem at critical points of the child welfare
process. Caseworkers interact with algorithms in their daily lives where they
must collect information about families and feed it to algorithms to make
critical decisions. We show how the interplay between systemic mechanics and
algorithmic decision-making can adversely impact the fairness of the
decision-making process itself. We show how functionality issues in algorithmic
systems can lead to process-oriented harms where they adversely affect the
nature of professional practice, and administration at the agency, and lead to
inconsistent and unreliable decisions at the street level. In addition,
caseworkers are compelled to undertake additional labor in the form of repair
work to restore disrupted administrative processes and decision-making, all
while facing organizational pressures and time and resource constraints.
Finally, we share the case study of a simple algorithmic tool that centers
caseworkers' decision-making within a trauma-informed framework and leads to
better outcomes, however, required a significant amount of investments on the
agency's part in creating the ecosystem for its proper use.
