  We address the problem of incrementally modeling and forecasting long-term
goals of a first-person camera wearer: what the user will do, where they will
go, and what goal they seek. In contrast to prior work in trajectory
forecasting, our algorithm, DARKO, goes further to reason about semantic states
(will I pick up an object?), and future goal states that are far in terms of
both space and time. DARKO learns and forecasts from first-person visual
observations of the user's daily behaviors via an Online Inverse Reinforcement
Learning (IRL) approach. Classical IRL discovers only the rewards in a batch
setting, whereas DARKO discovers the states, transitions, rewards, and goals of
a user from streaming data. Among other results, we show DARKO forecasts goals
better than competing methods in both noisy and ideal settings, and our
approach is theoretically and empirically no-regret.
