  Recognizing Activities of Daily Living (ADL) is a vital process for
intelligent assistive robots, but collecting large annotated datasets requires
time-consuming temporal labeling and raises privacy concerns, e.g., if the data
is collected in a real household. In this work, we explore the concept of
constructing training examples for ADL recognition by playing life simulation
video games and introduce the SIMS4ACTION dataset created with the popular
commercial game THE SIMS 4. We build Sims4Action by specifically executing
actions-of-interest in a "top-down" manner, while the gaming circumstances
allow us to freely switch between environments, camera angles and subject
appearances. While ADL recognition on gaming data is interesting from the
theoretical perspective, the key challenge arises from transferring it to the
real-world applications, such as smart-homes or assistive robotics. To meet
this requirement, Sims4Action is accompanied with a GamingToReal benchmark,
where the models are evaluated on real videos derived from an existing ADL
dataset. We integrate two modern algorithms for video-based activity
recognition in our framework, revealing the value of life simulation video
games as an inexpensive and far less intrusive source of training data.
However, our results also indicate that tasks involving a mixture of gaming and
real data are challenging, opening a new research direction. We will make our
dataset publicly available at https://github.com/aroitberg/sims4action.
