  When applying reinforcement learning (RL) to a new problem, reward
engineering is a necessary, but often difficult and error-prone task a system
designer has to face. To avoid this step, we propose LR4GPM, a novel (deep) RL
method that can optimize a global performance metric, which is supposed to be
available as part of the problem description. LR4GPM alternates between two
phases: (1) learning a (possibly vector) reward function used to fit the
performance metric, and (2) training a policy to optimize an approximation of
this performance metric based on the learned rewards. Such RL training is not
straightforward since both the reward function and the policy are trained using
non-stationary data. To overcome this issue, we propose several training
tricks. We demonstrate the efficiency of LR4GPM on several domains. Notably,
LR4GPM outperforms the winner of a recent autonomous driving competition
organized at DAI'2020.
