  Cooperative multi-agent learning methods are essential in developing
effective cooperation strategies in multi-agent domains. In robotics, these
methods extend beyond multi-robot scenarios to single-robot systems, where they
enable coordination among different robot modules (e.g., robot legs or joints).
However, current methods often struggle to quickly adapt to unforeseen
failures, such as a malfunctioning robot leg, especially after the algorithm
has converged to a strategy. To overcome this, we introduce the Relational
Q-Functionals (RQF) framework. RQF leverages a relational network, representing
agents' relationships, to enhance adaptability, providing resilience against
malfunction(s). Our algorithm also efficiently handles continuous state-action
domains, making it adept for robotic learning tasks. Our empirical results show
that RQF enables agents to use these relationships effectively to facilitate
cooperation and recover from an unexpected malfunction in single-robot systems
with multiple interacting modules. Thus, our approach offers promising
applications in multi-agent systems, particularly in scenarios with unforeseen
malfunctions.
