  How can we plan efficiently in real time to control an agent in a complex
environment that may involve many other agents? While existing sample-based
planners have enjoyed empirical success in large POMDPs, their performance
heavily relies on a fast simulator. However, real-world scenarios are complex
in nature and their simulators are often computationally demanding, which
severely limits the performance of online planners. In this work, we propose
influence-augmented online planning, a principled method to transform a
factored simulator of the entire environment into a local simulator that
samples only the state variables that are most relevant to the observation and
reward of the planning agent and captures the incoming influence from the rest
of the environment using machine learning methods. Our main experimental
results show that planning on this less accurate but much faster local
simulator with POMCP leads to higher real-time planning performance than
planning on the simulator that models the entire environment.
