  Identifying controllable aspects of the environment has proven to be an
extraordinary intrinsic motivator to reinforcement learning agents. Despite
repeatedly achieving State-of-the-Art results, this approach has only been
studied as a proxy to a reward-based task and has not yet been evaluated on its
own. Current methods are based on action-prediction. Humans, on the other hand,
assign blame to their actions to decide what they controlled. This work
proposes Controlled Effect Network (CEN), an unsupervised method based on
counterfactual measures of blame to identify effects on the environment
controlled by the agent. CEN is evaluated in a wide range of environments
showing that it can accurately identify controlled effects. Moreover, we
demonstrate CEN's capabilities as intrinsic motivator by integrating it in the
state-of-the-art exploration method, achieving substantially better performance
than action-prediction models.
