  Contextual bandits are online learners that, given an input, select an arm
and receive a reward for that arm. They use the reward as a learning signal and
aim to maximize the total reward over the inputs. Contextual bandits are
commonly used to solve recommendation or ranking problems. This paper considers
a learning setting in which multiple parties aim to train a contextual bandit
together in a private way: the parties aim to maximize the total reward but do
not want to share any of the relevant information they possess with the other
parties. Specifically, multiple parties have access to (different) features
that may benefit the learner but that cannot be shared with other parties. One
of the parties pulls the arm but other parties may not learn which arm was
pulled. One party receives the reward but the other parties may not learn the
reward value. This paper develops a privacy-preserving multi-party contextual
bandit for this learning setting by combining secure multi-party computation
with a differentially private mechanism based on epsilon-greedy exploration.
