  Most supervised learning models are trained for full automation. However,
their predictions are sometimes worse than those by human experts on some
specific instances. Motivated by this empirical observation, our goal is to
design classifiers that are optimized to operate under different automation
levels. More specifically, we focus on convex margin-based classifiers and
first show that the problem is NP-hard. Then, we further show that, for support
vector machines, the corresponding objective function can be expressed as the
difference of two functions f = g - c, where g is monotone, non-negative and
{\gamma}-weakly submodular, and c is non-negative and modular. This
representation allows a recently introduced deterministic greedy algorithm, as
well as a more efficient randomized variant of the algorithm, to enjoy
approximation guarantees at solving the problem. Experiments on synthetic and
real-world data from several applications in medical diagnosis illustrate our
theoretical findings and demonstrate that, under human assistance, supervised
learning models trained to operate under different automation levels can
outperform those trained for full automation as well as humans operating alone.
