  Knowledge-based systems have been used to monitor machines and processes in
the real world. In this paper we propose the use of knowledge-based systems to
monitor other AI systems in operation. We motivate and provide a problem
analysis of this novel setting and subsequently propose a framework that allows
for structuring future research related to this setting. Several directions for
further research are also discussed.
