  We present an approach for safe trajectory planning, where a strategic task
related to autonomous racing is learned sample-efficient within a simulation
environment. A high-level policy, represented as a neural network, outputs a
reward specification that is used within the cost function of a parametric
nonlinear model predictive controller (NMPC). By including constraints and
vehicle kinematics in the NLP, we are able to guarantee safe and feasible
trajectories related to the used model. Compared to classical reinforcement
learning (RL), our approach restricts the exploration to safe trajectories,
starts with a good prior performance and yields full trajectories that can be
passed to a tracking lowest-level controller. We do not address the
lowest-level controller in this work and assume perfect tracking of feasible
trajectories. We show the superior performance of our algorithm on simulated
racing tasks that include high-level decision making. The vehicle learns to
efficiently overtake slower vehicles and to avoid getting overtaken by blocking
faster vehicles.
