  Thompson sampling (TS) is a class of algorithms for sequential
decision-making, which requires maintaining a posterior distribution over a
model. However, calculating exact posterior distributions is intractable for
all but the simplest models. Consequently, efficient computation of an
approximate posterior distribution is a crucial problem for scalable TS with
complex models, such as neural networks. In this paper, we use distribution
optimization techniques to approximate the posterior distribution, solved via
Wasserstein gradient flows. Based on the framework, a principled
particle-optimization algorithm is developed for TS to approximate the
posterior efficiently. Our approach is scalable and does not make explicit
distribution assumptions on posterior approximations. Extensive experiments on
both synthetic data and real large-scale data demonstrate the superior
performance of the proposed methods.
