  An open research question in deep reinforcement learning is how to focus the
policy learning of key decisions within a sparse domain. This paper emphasizes
combining the advantages of inputoutput hidden Markov models and reinforcement
learning towards interpretable maintenance decisions. We propose a novel
hierarchical-modeling methodology that, at a high level, detects and interprets
the root cause of a failure as well as the health degradation of the turbofan
engine, while, at a low level, it provides the optimal replacement policy. It
outperforms the baseline performance of deep reinforcement learning methods
applied directly to the raw data or when using a hidden Markov model without
such a specialized hierarchy. It also provides comparable performance to prior
work, however, with the additional benefit of interpretability.
