Scalable Online Planning via Reinforcement Learning Fine-Tuning
  Lookahead search has been a critical component of recent AI successes, such
as in the games of chess, go, and poker. However, the search methods used in
these games, and in many other settings, are tabular. Tabular search methods do
not scale well with the size of the search space, and this problem is
exacerbated by stochasticity and partial observability. In this work we replace
tabular search with online model-based fine-tuning of a policy neural network
via reinforcement learning, and show that this approach outperforms
state-of-the-art search algorithms in benchmark settings. In particular, we use
our search algorithm to achieve a new state-of-the-art result in self-play
Hanabi, and show the generality of our algorithm by also showing that it
outperforms tabular search in the Atari game Ms. Pacman.
