  In the criminal justice system, algorithmic risk assessment instruments are
used to predict the risk a defendant poses to society; examples include the
risk of recidivating or the risk of failing to appear at future court dates.
However, defendants are also at risk of harm from the criminal justice system.
To date, there exists no risk assessment instrument that considers the risk the
system poses to the individual. We develop a risk assessment instrument that
"flips the script." Using data about U.S. federal sentencing decisions, we
build a risk assessment instrument that predicts the likelihood an individual
will receive an especially lengthy sentence given factors that should be
legally irrelevant to the sentencing decision. To do this, we develop a
two-stage modeling approach. Our first-stage model is used to determine which
sentences were "especially lengthy." We then use a second-stage model to
predict the defendant's risk of receiving a sentence that is flagged as
especially lengthy given factors that should be legally irrelevant. The factors
that should be legally irrelevant include, for example, race, court location,
and other socio-demographic information about the defendant. Our instrument
achieves comparable predictive accuracy to risk assessment instruments used in
pretrial and parole contexts. We discuss the limitations of our modeling
approach and use the opportunity to highlight how traditional risk assessment
instruments in various criminal justice settings also suffer from many of the
same limitations and embedded value systems of their creators.
