  Prediction APIs offered for a fee are a fast-growing industry and an
important part of machine learning as a service. While many such services are
available, the heterogeneity in their price and performance makes it
challenging for users to decide which API or combination of APIs to use for
their own data and budget. We take a first step towards addressing this
challenge by proposing FrugalML, a principled framework that jointly learns the
strength and weakness of each API on different data, and performs an efficient
optimization to automatically identify the best sequential strategy to
adaptively use the available APIs within a budget constraint. Our theoretical
analysis shows that natural sparsity in the formulation can be leveraged to
make FrugalML efficient. We conduct systematic experiments using ML APIs from
Google, Microsoft, Amazon, IBM, Baidu and other providers for tasks including
facial emotion recognition, sentiment analysis and speech recognition. Across
various tasks, FrugalML can achieve up to 90% cost reduction while matching the
accuracy of the best single API, or up to 5% better accuracy while matching the
best API's cost.
