  In this paper, we present our vision of differentiable ML pipelines called
DiffML to automate the construction of ML pipelines in an end-to-end fashion.
The idea is that DiffML allows to jointly train not just the ML model itself
but also the entire pipeline including data preprocessing steps, e.g., data
cleaning, feature selection, etc. Our core idea is to formulate all pipeline
steps in a differentiable way such that the entire pipeline can be trained
using backpropagation. However, this is a non-trivial problem and opens up many
new research questions. To show the feasibility of this direction, we
demonstrate initial ideas and a general principle of how typical preprocessing
steps such as data cleaning, feature selection and dataset selection can be
formulated as differentiable programs and jointly learned with the ML model.
Moreover, we discuss a research roadmap and core challenges that have to be
systematically tackled to enable fully differentiable ML pipelines.
