  Machine learning models can make critical errors that are easily hidden
within vast amounts of data. Such errors often run counter to rules based on
human intuition. However, rules based on human knowledge are challenging to
scale or to even formalize. We thereby seek to infer statistical rules from the
data and quantify the extent to which a model has learned them. We propose a
framework SQRL that integrates logic-based methods with statistical inference
to derive these rules from a model's training data without supervision. We
further show how to adapt models at test time to reduce rule violations and
produce more coherent predictions. SQRL generates up to 300K rules over
datasets from vision, tabular, and language settings. We uncover up to 158K
violations of those rules by state-of-the-art models for classification, object
detection, and data imputation. Test-time adaptation reduces these violations
by up to 68.7% with relative performance improvement up to 32%. SQRL is
available at https://github.com/DebugML/sqrl.
