  In this paper, we consider decentralized sequential decision making in
distributed online recommender systems, where items are recommended to users
based on their search query as well as their specific background including
history of bought items, gender and age, all of which comprise the context
information of the user. In contrast to centralized recommender systems, in
which there is a single centralized seller who has access to the complete
inventory of items as well as the complete record of sales and user
information, in decentralized recommender systems each seller/learner only has
access to the inventory of items and user information for its own products and
not the products and user information of other sellers, but can get commission
if it sells an item of another seller. Therefore the sellers must distributedly
find out for an incoming user which items to recommend (from the set of own
items or items of another seller), in order to maximize the revenue from own
sales and commissions. We formulate this problem as a cooperative contextual
bandit problem, analytically bound the performance of the sellers compared to
the best recommendation strategy given the complete realization of user
arrivals and the inventory of items, as well as the context-dependent purchase
probabilities of each item, and verify our results via numerical examples on a
distributed data set adapted based on Amazon data. We evaluate the dependence
of the performance of a seller on the inventory of items the seller has, the
number of connections it has with the other sellers, and the commissions which
the seller gets by selling items of other sellers to its users.
