  Artificial intelligence (AI) and machine learning (ML) are increasingly
broadly adopted in industry, However, based on well over a dozen case studies,
we have learned that deploying industry-strength, production quality ML models
in systems proves to be challenging. Companies experience challenges related to
data quality, design methods and processes, performance of models as well as
deployment and compliance. We learned that a new, structured engineering
approach is required to construct and evolve systems that contain ML/DL
components. In this paper, we provide a conceptualization of the typical
evolution patterns that companies experience when employing ML as well as an
overview of the key problems experienced by the companies that we have studied.
The main contribution of the paper is a research agenda for AI engineering that
provides an overview of the key engineering challenges surrounding ML solutions
and an overview of open items that need to be addressed by the research
community at large.
