  ML APIs have greatly relieved application developers of the burden to design
and train their own neural network models -- classifying objects in an image
can now be as simple as one line of Python code to call an API. However, these
APIs offer the same pre-trained models regardless of how their output is used
by different applications. This can be suboptimal as not all ML inference
errors can cause application failures, and the distinction between inference
errors that can or cannot cause failures varies greatly across applications.
  To tackle this problem, we first study 77 real-world applications, which
collectively use six ML APIs from two providers, to reveal common patterns of
how ML API output affects applications' decision processes. Inspired by the
findings, we propose ChameleonAPI, an optimization framework for ML APIs, which
takes effect without changing the application source code. ChameleonAPI
provides application developers with a parser that automatically analyzes the
application to produce an abstract of its decision process, which is then used
to devise an application-specific loss function that only penalizes API output
errors critical to the application. ChameleonAPI uses the loss function to
efficiently train a neural network model customized for each application and
deploys it to serve API invocations from the respective application via
existing interface. Compared to a baseline that selects the best-of-all
commercial ML API, we show that ChameleonAPI reduces incorrect application
decisions by 43%.
