  With the emergence of Virtual and Mixed Reality (XR) devices, eye tracking
has received significant attention in the computer vision community. Eye gaze
estimation is a crucial component in XR -- enabling energy efficient rendering,
multi-focal displays, and effective interaction with content. In head-mounted
XR devices, the eyes are imaged off-axis to avoid blocking the field of view.
This leads to increased challenges in inferring eye related quantities and
simultaneously provides an opportunity to develop accurate and robust learning
based approaches. To this end, we present MagicEyes, the first large scale eye
dataset collected using real MR devices with comprehensive ground truth
labeling. MagicEyes includes $587$ subjects with $80,000$ images of
human-labeled ground truth and over $800,000$ images with gaze target labels.
We evaluate several state-of-the-art methods on MagicEyes and also propose a
new multi-task EyeNet model designed for detecting the cornea, glints and pupil
along with eye segmentation in a single forward pass.
