  In the Machine Learning (ML) model development lifecycle, training candidate
models using an offline holdout dataset and identifying the best model for the
given task is only the first step. After the deployment of the selected model,
continuous model monitoring and model retraining is required in many real-world
applications. There are multiple reasons for retraining, including data or
concept drift, which may be reflected on the model performance as monitored by
an appropriate metric. Another motivation for retraining is the acquisition of
increasing amounts of data over time, which may be used to retrain and improve
the model performance even in the absence of drifts. We examine the impact of
various retraining decision points on crucial factors, such as model
performance and resource utilization, in the context of Multilabel
Classification models. We explain our key decision points and propose a
reference framework for designing an effective model retraining strategy.
