Online Planning Algorithms for POMDPs
  Partially Observable Markov Decision Processes (POMDPs) provide a rich
framework for sequential decision-making under uncertainty in stochastic
domains. However, solving a POMDP is often intractable except for small
problems due to their complexity. Here, we focus on online approaches that
alleviate the computational complexity by computing good local policies at each
decision step during the execution. Online algorithms generally consist of a
lookahead search to find the best action to execute at each time step in an
environment. Our objectives here are to survey the various existing online
POMDP methods, analyze their properties and discuss their advantages and
disadvantages; and to thoroughly evaluate these online approaches in different
environments under various metrics (return, error bound reduction, lower bound
improvement). Our experimental results indicate that state-of-the-art online
heuristic search methods can handle large POMDP domains efficiently.
