  The development and deployment of machine learning (ML) systems can be
executed easily with modern tools, but the process is typically rushed and
means-to-an-end. The lack of diligence can lead to technical debt, scope creep
and misaligned objectives, model misuse and failures, and expensive
consequences. Engineering systems, on the other hand, follow well-defined
processes and testing standards to streamline development for high-quality,
reliable results. The extreme is spacecraft systems, where mission critical
measures and robustness are ingrained in the development process. Drawing on
experience in both spacecraft engineering and ML (from research through product
across domain areas), we have developed a proven systems engineering approach
for machine learning development and deployment. Our "Machine Learning
Technology Readiness Levels" (MLTRL) framework defines a principled process to
ensure robust, reliable, and responsible systems while being streamlined for ML
workflows, including key distinctions from traditional software engineering.
Even more, MLTRL defines a lingua franca for people across teams and
organizations to work collaboratively on artificial intelligence and machine
learning technologies. Here we describe the framework and elucidate it with
several real world use-cases of developing ML methods from basic research
through productization and deployment, in areas such as medical diagnostics,
consumer computer vision, satellite imagery, and particle physics.
