  Designing activity detection systems that can be successfully deployed in
daily-living environments requires datasets that pose the challenges typical of
real-world scenarios. In this paper, we introduce a new untrimmed daily-living
dataset that features several real-world challenges: Toyota Smarthome Untrimmed
(TSU). TSU contains a wide variety of activities performed in a spontaneous
manner. The dataset contains dense annotations including elementary, composite
activities and activities involving interactions with objects. We provide an
analysis of the real-world challenges featured by our dataset, highlighting the
open issues for detection algorithms. We show that current state-of-the-art
methods fail to achieve satisfactory performance on the TSU dataset. Therefore,
we propose a new baseline method for activity detection to tackle the novel
challenges provided by our dataset. This method leverages one modality (i.e.
optic flow) to generate the attention weights to guide another modality (i.e
RGB) to better detect the activity boundaries. This is particularly beneficial
to detect activities characterized by high temporal variance. We show that the
method we propose outperforms state-of-the-art methods on TSU and on another
popular challenging dataset, Charades.
