  Learning effective policies for real-world problems is still an open
challenge for the field of reinforcement learning (RL). The main limitation
being the amount of data needed and the pace at which that data can be
obtained. In this paper, we study how to build lightweight simulators of
complicated systems that can run sufficiently fast for deep RL to be
applicable. We focus on domains where agents interact with a reduced portion of
a larger environment while still being affected by the global dynamics. Our
method combines the use of local simulators with learned models that mimic the
influence of the global system. The experiments reveal that incorporating this
idea into the deep RL workflow can considerably accelerate the training process
and presents several opportunities for the future.
