  In this paper, we investigate the practical relevance of explainable
artificial intelligence (XAI) with a special focus on the producing industries
and relate them to the current state of academic XAI research. Our findings are
based on an extensive series of interviews regarding the role and applicability
of XAI along the Machine Learning (ML) lifecycle in current industrial practice
and its expected relevance in the future. The interviews were conducted among a
great variety of roles and key stakeholders from different industry sectors. On
top of that, we outline the state of XAI research by providing a concise review
of the relevant literature. This enables us to provide an encompassing overview
covering the opinions of the surveyed persons as well as the current state of
academic research. By comparing our interview results with the current research
approaches we reveal several discrepancies. While a multitude of different XAI
approaches exists, most of them are centered around the model evaluation phase
and data scientists. Their versatile capabilities for other stages are
currently either not sufficiently explored or not popular among practitioners.
In line with existing work, our findings also confirm that more efforts are
needed to enable also non-expert users' interpretation and understanding of
opaque AI models with existing methods and frameworks.
