  We investigate the feasibility of learning from a mix of both fully-labeled
supervised data and contextual bandit data. We specifically consider settings
in which the underlying learning signal may be different between these two data
sources. Theoretically, we state and prove no-regret algorithms for learning
that is robust to misaligned cost distributions between the two sources.
Empirically, we evaluate some of these algorithms on a large selection of
datasets, showing that our approach is both feasible and helpful in practice.
