  Real-world applications of machine learning models are often subject to legal
or policy-based regulations. Some of these regulations require ensuring the
validity of the model, i.e., the approximation error being smaller than a
threshold. A global metric is generally too insensitive to determine the
validity of a specific prediction, whereas evaluating local validity is costly
since it requires gathering additional data.We propose learning the model error
to acquire a local validity estimate while reducing the amount of required data
through active learning. Using model validation benchmarks, we provide
empirical evidence that the proposed method can lead to an error model with
sufficient discriminative properties using a relatively small amount of data.
Furthermore, an increased sensitivity to local changes of the validity bounds
compared to alternative approaches is demonstrated.
