  A successful tactic that is followed by the scientific community for
advancing AI is to treat games as problems, which has been proven to lead to
various breakthroughs. We adapt this strategy in order to study Rocket League,
a widely popular but rather under-explored 3D multiplayer video game with a
distinct physics engine and complex dynamics that pose a significant challenge
in developing efficient and high-performance game-playing agents. In this
paper, we present Lucy-SKG, a Reinforcement Learning-based model that learned
how to play Rocket League in a sample-efficient manner, outperforming by a
notable margin the two highest-ranking bots in this game, namely Necto (2022
bot champion) and its successor Nexto, thus becoming a state-of-the-art agent.
Our contributions include: a) the development of a reward analysis and
visualization library, b) novel parameterizable reward shape functions that
capture the utility of complex reward types via our proposed Kinesthetic Reward
Combination (KRC) technique, and c) design of auxiliary neural architectures
for training on reward prediction and state representation tasks in an
on-policy fashion for enhanced efficiency in learning speed and performance. By
performing thorough ablation studies for each component of Lucy-SKG, we showed
their independent effectiveness in overall performance. In doing so, we
demonstrate the prospects and challenges of using sample-efficient
Reinforcement Learning techniques for controlling complex dynamical systems
under competitive team-based multiplayer conditions.
