  The plethora of sensors in our commodity devices provides a rich substrate
for sensor-fused tracking. Yet, today's solutions are unable to deliver robust
and high tracking accuracies across multiple agents in practical, everyday
environments - a feature central to the future of immersive and collaborative
applications. This can be attributed to the limited scope of diversity
leveraged by these fusion solutions, preventing them from catering to the
multiple dimensions of accuracy, robustness (diverse environmental conditions)
and scalability (multiple agents) simultaneously. In this work, we take an
important step towards this goal by introducing the notion of dual-layer
diversity to the problem of sensor fusion in multi-agent tracking. We
demonstrate that the fusion of complementary tracking modalities, -
passive/relative (e.g., visual odometry) and active/absolute tracking (e.g.,
infrastructure-assisted RF localization) offer a key first layer of diversity
that brings scalability while the second layer of diversity lies in the
methodology of fusion, where we bring together the complementary strengths of
algorithmic (for robustness) and data-driven (for accuracy) approaches. RoVaR
is an embodiment of such a dual-layer diversity approach that intelligently
attends to cross-modal information using algorithmic and data-driven techniques
that jointly share the burden of accurately tracking multiple agents in the
wild. Extensive evaluations reveal RoVaR's multi-dimensional benefits in terms
of tracking accuracy (median of 15cm), robustness (in unseen environments),
light weight (runs in real-time on mobile platforms such as Jetson Nano/TX2),
to enable practical multi-agent immersive applications in everyday
environments.
