  A significant amount of people die in road accidents due to driver errors. To
reduce fatalities, developing intelligent driving systems assisting drivers to
identify potential risks is in an urgent need. Risky situations are generally
defined based on collision prediction in the existing works. However, collision
is only a source of potential risks, and a more generic definition is required.
In this work, we propose a novel driver-centric definition of risk, i.e.,
objects influencing drivers' behavior are risky. A new task called risk object
identification is introduced. We formulate the task as the cause-effect problem
and present a novel two-stage risk object identification framework based on
causal inference with the proposed object-level manipulable driving model. We
demonstrate favorable performance on risk object identification compared with
strong baselines on the Honda Research Institute Driving Dataset (HDD). Our
framework achieves a substantial average performance boost over a strong
baseline by 7.5%.
