  In spite of machine learning's rapid growth, its engineering support is
scattered in many forms, and tends to favor certain engineering stages,
stakeholders, and evaluation preferences. We envision a capability-based
framework, which uses fine-grained specifications for ML model behaviors to
unite existing efforts towards better ML engineering. We use concrete scenarios
(model design, debugging, and maintenance) to articulate capabilities' broad
applications across various different dimensions, and their impact on building
safer, more generalizable and more trustworthy models that reflect human needs.
Through preliminary experiments, we show capabilities' potential for reflecting
model generalizability, which can provide guidance for ML engineering process.
We discuss challenges and opportunities for capabilities' integration into ML
engineering.
