  Enforcing safety for dynamical systems is challenging, since it requires
constraint satisfaction along trajectory predictions. Equivalent control
constraints can be computed in the form of sets that enforce positive
invariance, and can thus guarantee safety in feedback controllers without
predictions. However, these constraints are cumbersome to compute from models,
and it is not yet well established how to infer constraints from data. In this
paper, we shed light on the key objects involved in learning control
constraints from data in a model-free setting. In particular, we discuss the
family of constraints that enforce safety in the context of a nominal control
policy, and expose that these constraints do not need to be accurate
everywhere. They only need to correctly exclude a subset of the state-actions
that would cause failure, which we call the critical set.
