  Modern applications are increasingly driven by Machine Learning (ML) models
whose non-deterministic behavior is affecting the entire application life cycle
from design to operation. The pervasive adoption of ML is urgently calling for
approaches that guarantee a stable non-functional behavior of ML-based
applications over time and across model changes. To this aim, non-functional
properties of ML models, such as privacy, confidentiality, fairness, and
explainability, must be monitored, verified, and maintained. Existing
approaches mostly focus on i) implementing solutions for classifier selection
according to the functional behavior of ML models, ii) finding new algorithmic
solutions, such as continuous re-training. In this paper, we propose a
multi-model approach that aims to guarantee a stable non-functional behavior of
ML-based applications. An architectural and methodological approach is provided
to compare multiple ML models showing similar non-functional properties and
select the model supporting stable non-functional behavior over time according
to (dynamic and unpredictable) contextual changes. Our approach goes beyond the
state of the art by providing a solution that continuously guarantees a stable
non-functional behavior of ML-based applications, is ML algorithm-agnostic, and
is driven by non-functional properties assessed on the ML models themselves. It
consists of a two-step process working during application operation, where
model assessment verifies non-functional properties of ML models trained and
selected at development time, and model substitution guarantees continuous and
stable support of non-functional properties. We experimentally evaluate our
solution in a real-world scenario focusing on non-functional property fairness.
