  Systems for making determinations on socially-constructed and complex
concepts at scale are increasingly being deployed. To make such fuzzy concepts
tractable for training and evaluating AI, aligning model outputs, or
human-in-the-loop workflows, the prevailing strategy involves developing
`constitutions' in the form of rules, policies, or principles. However,
high-level rules often fail to capture situational nuances or have differing
interpretations, resulting in inconsistent decisions. In this work, we
introduce case law grounding (CLG), a hybrid workflow inspired by case law in
the legal realm where past judgments on specific cases inform new decisions.
Evaluating on two task domains, we find that CLG can improve alignment of
decisions (+9.6% and +10.9% accuracy) and consistency ($\Delta\bar{\kappa}$ of
+0.263 and +0.433) of human decision-makers, while also providing auditable
rationales. We also find similarly substantial alignment improvements for an
LLM decision-maker (+25% and +23% accuracy).
