  When environmental interaction is expensive, model-based reinforcement
learning offers a solution by planning ahead and avoiding costly mistakes.
Model-based agents typically learn a single-step transition model. In this
paper, we propose a multi-step model that predicts the outcome of an action
sequence with variable length. We show that this model is easy to learn, and
that the model can make policy-conditional predictions. We report preliminary
results that show a clear advantage for the multi-step model compared to its
one-step counterpart.
