  We introduce Nymeria - a large-scale, diverse, richly annotated human motion
dataset collected in the wild with multiple multimodal egocentric devices. The
dataset comes with a) full-body 3D motion ground truth; b) egocentric
multimodal recordings from Project Aria devices with RGB, grayscale,
eye-tracking cameras, IMUs, magnetometer, barometer, and microphones; and c) an
additional "observer" device providing a third-person viewpoint. We compute
world-aligned 6DoF transformations for all sensors, across devices and capture
sessions. The dataset also provides 3D scene point clouds and calibrated gaze
estimation. We derive a protocol to annotate hierarchical language descriptions
of in-context human motion, from fine-grain pose narrations, to atomic actions
and activity summarization. To the best of our knowledge, the Nymeria dataset
is the world largest in-the-wild collection of human motion with natural and
diverse activities; first of its kind to provide synchronized and localized
multi-device multimodal egocentric data; and the world largest dataset with
motion-language descriptions. It contains 1200 recordings of 300 hours of daily
activities from 264 participants across 50 locations, travelling a total of
399Km. The motion-language descriptions provide 310.5K sentences in 8.64M words
from a vocabulary size of 6545. To demonstrate the potential of the dataset we
define key research tasks for egocentric body tracking, motion synthesis, and
action recognition and evaluate several state-of-the-art baseline algorithms.
Data and code will be open-sourced.
