  The machine learning lifecycle extends beyond the deployment stage.
Monitoring deployed models is crucial for continued provision of high quality
machine learning enabled services. Key areas include model performance and data
monitoring, detecting outliers and data drift using statistical techniques, and
providing explanations of historic predictions. We discuss the challenges to
successful implementation of solutions in each of these areas with some recent
examples of production ready solutions using open source tools.
