  It has recently been shown that reinforcement learning can be used to train
generators capable of producing high-quality game levels, with quality defined
in terms of some user-specified heuristic. To ensure that these generators'
output is sufficiently diverse (that is, not amounting to the reproduction of a
single optimal level configuration), the generation process is constrained such
that the initial seed results in some variance in the generator's output.
However, this results in a loss of control over the generated content for the
human user. We propose to train generators capable of producing controllably
diverse output, by making them "goal-aware." To this end, we add conditional
inputs representing how close a generator is to some heuristic, and also modify
the reward mechanism to incorporate that value. Testing on multiple domains, we
show that the resulting level generators are capable of exploring the space of
possible levels in a targeted, controllable manner, producing levels of
comparable quality as their goal-unaware counterparts, that are diverse along
designer-specified dimensions.
