  Hierarchical decomposition of control is unavoidable in large dynamical
systems. In reinforcement learning (RL), it is usually solved with subgoals
defined at higher policy levels and achieved at lower policy levels. Reaching
these goals can take a substantial amount of time, during which it is not
verified whether they are still worth pursuing. However, due to the randomness
of the environment, these goals may become obsolete. In this paper, we address
this gap in the state-of-the-art approaches and propose a method in which the
validity of higher-level actions (thus lower-level goals) is constantly
verified at the higher level. If the actions, i.e. lower level goals, become
inadequate, they are replaced by more appropriate ones. This way we combine the
advantages of hierarchical RL, which is fast training, and flat RL, which is
immediate reactivity. We study our approach experimentally on seven benchmark
environments.
