  Deploying teams of unmanned aerial vehicles (UAVs) to harvest data from
distributed Internet of Things (IoT) devices requires efficient trajectory
planning and coordination algorithms. Multi-agent reinforcement learning (MARL)
has emerged as a solution, but requires extensive and costly real-world
training data. To tackle this challenge, we propose a novel model-aided
federated MARL algorithm to coordinate multiple UAVs on a data harvesting
mission with only limited knowledge about the environment. The proposed
algorithm alternates between building an environment simulation model from
real-world measurements, specifically learning the radio channel
characteristics and estimating unknown IoT device positions, and federated QMIX
training in the simulated environment. Each UAV agent trains a local QMIX model
in its simulated environment and continuously consolidates it through federated
learning with other agents, accelerating the learning process. A performance
comparison with standard MARL algorithms demonstrates that our proposed
model-aided FedQMIX algorithm reduces the need for real-world training
experiences by around three magnitudes while attaining similar data collection
performance.
