  We routinely perform procedures (such as cooking) that include a set of
atomic steps. Often, inadvertent omission or misordering of a single step can
lead to serious consequences, especially for those experiencing cognitive
challenges such as dementia. This paper introduces PrISM-Observer, a
smartwatch-based, context-aware, real-time intervention system designed to
support daily tasks by preventing errors. Unlike traditional systems that
require users to seek out information, the agent observes user actions and
intervenes proactively. This capability is enabled by the agent's ability to
continuously update its belief in the user's behavior in real-time through
multimodal sensing and forecast optimal intervention moments and methods. We
first validated the steps-tracking performance of our framework through
evaluations across three datasets with different complexities. Then, we
implemented a real-time agent system using a smartwatch and conducted a user
study in a cooking task scenario. The system generated helpful interventions,
and we gained positive feedback from the participants. The general
applicability of PrISM-Observer to daily tasks promises broad applications, for
instance, including support for users requiring more involved interventions,
such as people with dementia or post-surgical patients.
