  In the context of addressing the Robot Air Hockey Challenge 2023, we
investigate the applicability of model-based deep reinforcement learning to
acquire a policy capable of autonomously playing air hockey. Our agents learn
solely from sparse rewards while incorporating self-play to iteratively refine
their behaviour over time. The robotic manipulator is interfaced using
continuous high-level actions for position-based control in the Cartesian plane
while having partial observability of the environment with stochastic
transitions. We demonstrate that agents are prone to overfitting when trained
solely against a single playstyle, highlighting the importance of self-play for
generalization to novel strategies of unseen opponents. Furthermore, the impact
of the imagination horizon is explored in the competitive setting of the highly
dynamic game of air hockey, with longer horizons resulting in more stable
learning and better overall performance.
