  It is common practice to use large computational resources to train neural
networks, as is known from many examples, such as reinforcement learning
applications. However, while massively parallel computing is often used for
training models, it is rarely used for searching solutions for combinatorial
optimization problems. In this paper, we propose a novel massively parallel
Monte-Carlo Tree Search (MP-MCTS) algorithm that works efficiently for 1,000
worker scale, and apply it to molecular design. This is the first work that
applies distributed MCTS to a real-world and non-game problem. Existing work on
large-scale parallel MCTS show efficient scalability in terms of the number of
rollouts up to 100 workers, but suffer from the degradation in the quality of
the solutions. MP-MCTS maintains the search quality at larger scale, and by
running MP-MCTS on 256 CPU cores for only 10 minutes, we obtained candidate
molecules having similar score to non-parallel MCTS running for 42 hours.
Moreover, our results based on parallel MCTS (combined with a simple RNN model)
significantly outperforms existing state-of-the-art work. Our method is generic
and is expected to speed up other applications of MCTS.
