  Automatic detection of intake gestures is a key element of automatic dietary
monitoring. Several types of sensors, including inertial measurement units
(IMU) and video cameras, have been used for this purpose. The common machine
learning approaches make use of the labeled sensor data to automatically learn
how to make detections. One characteristic, especially for deep learning
models, is the need for large datasets. To meet this need, we collected the
Objectively Recognizing Eating Behavior and Associated Intake (OREBA) dataset.
The OREBA dataset aims to provide comprehensive multi-sensor data recorded
during the course of communal meals for researchers interested in intake
gesture detection. Two scenarios are included, with 100 participants for a
discrete dish and 102 participants for a shared dish, totalling 9069 intake
gestures. Available sensor data consists of synchronized frontal video and IMU
with accelerometer and gyroscope for both hands. We report the details of data
collection and annotation, as well as details of sensor processing. The results
of studies on IMU and video data involving deep learning models are reported to
provide a baseline for future research. Specifically, the best baseline models
achieve performances of $F_1$ = 0.853 for the discrete dish using video and
$F_1$ = 0.852 for the shared dish using inertial data.
