  In many real-world problems, there is the possibility to configure, to a
limited extent, some environmental parameters to improve the performance of a
learning agent. In this paper, we propose a novel framework, Configurable
Markov Decision Processes (Conf-MDPs), to model this new type of interaction
with the environment. Furthermore, we provide a new learning algorithm, Safe
Policy-Model Iteration (SPMI), to jointly and adaptively optimize the policy
and the environment configuration. After having introduced our approach and
derived some theoretical results, we present the experimental evaluation in two
explicative problems to show the benefits of the environment configurability on
the performance of the learned policy.
