VitrAI - Applying Explainable AI in the Real World
Marc Hanussek\inst{1
With recent progress in the field of Explainable Artificial Intelligence (XAI) and increasing use in practice, the need for an evaluation of different XAI methods and their explanation quality in practical usage scenarios arises. For this purpose, we present VitrAI, which is a web-based service with the goal of uniformly demonstrating four different XAI algorithms in the context of three real life scenarios and evaluating their performance and comprehensibility for humans. This work reveals practical obstacles when adopting XAI methods and gives qualitative estimates on how well different approaches perform in said scenarios. % We would like to encourage you to list your keywords within % the abstract section using the \keywords{... command. \keywords{Explainable Artificial Intelligence, XAI prototype, evaluation of explanations
paper.tex

%%%%%%%%%%%%%%%%%%%% author.tex %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% sample root file for your "contribution" to a proceedings volume
%
% Use this file as a template for your own input.
%
%%%%%%%%%%%%%%%% Springer %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


\documentclass{svproc}
%
% RECOMMENDED %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%

% to typeset URLs, URIs, and DOIs
\usepackage{url}
\def\UrlFont{\rmfamily}
\usepackage{amssymb}
\usepackage[hidelinks]{hyperref}
\usepackage{graphicx}


\begin{document}
\mainmatter              % start of a contribution
%
\title{VitrAI - Applying Explainable AI in the Real World}
%
\titlerunning{VitrAI - Applying Explainable AI in the Real World}  % abbreviated title (for running head)
%                                     also used for the TOC unless
%                                     \toctitle is used
%
\author{Marc Hanussek\inst{1} \and Falko Kötter\inst{2} \and Maximilien Kintz\inst{2} \and Jens Drawehn\inst{2}}
%
\authorrunning{Hanussek et al.} % abbreviated author list (for running head)
%

\institute{University of Stuttgart IAT, Institute of Human Factors and Technology Management, Stuttgart, Germany,\\
\email{marc.hanussek@iat.uni-stuttgart.de},\\ WWW home page:
\texttt{https://www.iat.uni-stuttgart.de/en/}
\and
Fraunhofer IAO, Fraunhofer Institute for Industrial Engineering IAO,\\
Stuttgart, Germany}

\maketitle              % typeset the title of the contribution

\begin{abstract}
With recent progress in the field of Explainable Artificial Intelligence (XAI) and increasing use in practice, the need for an evaluation of different XAI methods and their explanation quality in practical usage scenarios arises. For this purpose, we present \emph{VitrAI}, which is a web-based service with the goal of uniformly demonstrating four different XAI algorithms in the context of three real life scenarios and evaluating their performance and comprehensibility for humans. This work reveals practical obstacles when adopting XAI methods and gives qualitative estimates on how well different approaches perform in said scenarios.
% We would like to encourage you to list your keywords within
% the abstract section using the \keywords{...} command.
\keywords{Explainable Artificial Intelligence, XAI prototype, evaluation of explanations}
\end{abstract}
%
\section{Introduction}
\label{sec:introduction}
%

The successful adoption of Artificial Intelligence (AI) relies heavily on how well decision makers can understand and trust it~\cite{doshivelez2017rigorous}. In the near future, AI will make far reaching decisions about humans, for example in self-driving cars, for loan applications or in the criminal justice system. Examples like COMPAS, a racially biased algorithm for parole decisions show the need for reviewing AI decisions~\cite{dressel2018compas}.

But complex AI models like Deep Neural Networks (DNNs) are hard to understand for humans and act as de-facto black boxes~\cite{mittelstadt2019explainingexplanations}. In comparison, simpler models like decision trees are more understandable for humans, but lack prediction accuracy~\cite{gunning2017explainable}, though some argue this is not necessarily true~\cite{rudin2019stopblackbox}. The field of Explainable Artificial Intelligence aims to create more explainable models while still achieving high predictive power. Current discussions in research suggest that explainability will lead to increased trust, realistic expectations and fairer decisions~\cite{doshivelez2017rigorous}. 

\emph{Post-Hoc explanation techniques} work by creating a simplified, interpretable explanation model that approximates the behavior of the black box~\cite{slack2020foolinglime}. However, this simplification is not without risk, as by definition the precise workings of the black box are not shown~\cite{slack2020foolinglime}. In contrast, \emph{interpretable models} aim to make the actual model used for predictions more transparent~\cite{rudin2019stopblackbox}. Basic examples of interpretable models are decisions trees and Bayesian rules lists~\cite{letham2015interpretable}. Neural networks can be made interpretable by tracking the degree to which each part of the input data contributed to the decision~\cite{zhang2018interpretablenn}.

Though the term \emph{model interpretability} is widely used in literature, it lacks an agreed-upon definition~\cite{lipton2018mythosinterpretability}. The same is true for the term \emph{explanation}, in particular what is a valid explanation and how to measure explanation quality. This stems from the difference between causality and correlation. In the aforementioned COMPAS example, it is unclear if race is used as a decision criterion, but still correlates strongly with the decision~\cite{dressel2018compas}\cite{rudin2019stopblackbox}. In another example, the model differentiated between pictures of huskies and wolves by the presence of snow, as wild animals tend to be photographed in the wilderness. To a human, this may be a bad explanation, though it correctly reveals the inner workings of the model~\cite{10.1145/2939672.2939778}. Therefore, it is necessary to distinguish the quality of explanations according to different purposes pursued.

For \emph{data scientists}, explanation quality hinges on the degree with which the actual decision criteria are shown. A high-quality explanation for wrong decisions would immediately show the human that the solution is low-quality, ideally giving an indication of a systematic error (e.g. due to bias in the input data). For \emph{end users}, explanation quality hinges on the degree to which an AI decision can be assessed based on the explanation. For example, if an AI identifies business cases from letters, the most relevant keywords could be highlighted so the human can decide if the decision is correct or if a manual reclassification is necessary. An explanation of the model's workings is secondary in comparison to a decision basis for human validation.

Ideally, AI reasons in the same way as humans, so that the explanation can be understood by humans. But this is not necessarily the case. There are many problems that are hard for AI and easy for humans (and vice versa). One example is general image classification, e.g. of animals. To humans differentiating between cats and dogs is trivial and an explanation provides little value to end users. Thus, evaluating explanations must also take into account tasks that are non-trivial for humans as well, for example identifying brands of vehicles.

For the use of AI in practice, it is necessary to provide high-quality explanations for end users. Thus, for developers of AI-based solutions it is necessary to evaluate explanation quality from an end-user point of view. Further research is necessary in what constitutes explanation quality in practical use and how that quality can be systematically evaluated, and furthermore measured.

In this work we present \emph{VitrAI}, a platform for demonstrating and evaluating XAI. We outline the main sections of the platform and XAI methods. We present a preliminary evaluation of explanation quality for theses tasks and describe how we plan to systematically evaluate explanation quality and compare human and AI explanations.


\section{Related Work}
\label{sec:relatedwork}

In this section, we investigate related work in the areas of (a) defining explainability and related terms and (b) benchmarking explainability.

%Defining explainability

\cite{lipton2018mythosinterpretability} defines desiderata for interpretability: Trust, causality, transferability, informativeness, fair and ethical decision making. Interpretability can be achieved by transparency or post-hoc explanations. Model properties that confer transparency are simulability, decomposability and algorithmic transparency. Results may be explained post-hoc by examples, verbal reasoning, visualizations, or local explanations. The author stresses the importance of a specific definition and goal when measuring interpretability and cautions against limiting the predictive power of AI for interpretability.

\cite{zhou2020intelligibility} argue that interpretability is not an intrinsic property of a model, but a property in perspective to a specific audience: Engineers, users or affectees, each of which have different goals and requirements regarding interpretability, and interpretability needs to be measured in regards to these goals.

\cite{montavon2018methods} detail current efforts to measure intelligibility, for example introspective self-reports from people, questionnaires and propose quantitative measurements for evaluating explanation quality, in particular continuity and selectivity.

In~\cite{das2020xaisurvey} an explanation is defined in the context of XAI as ``a way to verify the output decision made by an AI agent or algorithm'', which corresponds to the explanation quality for end users we have outlined. In addition, this work gives an overview of desired qualities promoted by explanations (trust, transparency, fairness) and explains different techniques for evaluating XAI algorithms. 

%Benchmarking explainability

\cite{zhou2020intelligibility} outline possibilities for evaluating explainability, noting that human understanding of systems is implicit and that a benchmark would need to encompass enough experience for a human to build such an implicit mental model. They note however a lack of explicit measures of explainability. Interpretability is different from explanation accuracy, as falsehoods such as simplifications can aid overall understanding.

\cite{zhou2020assessingposthoc} propose leveraging existing work in the field of learning science by interpreting an AI's user as a learner and an explanation as learning content. They compare pairs of explanations in an educational context with user trials and semi-structured interviews. Using interview transcripts, they compared the user experience of the explanations. As postulated in~\cite{zhou2020intelligibility}, it was shown that helpfulness of different explanations depends on users' educational background.

\cite{geirhos2018humandnnbenchmark} investigates the capabilities of Deep Neural Networks to generalize by comparing DNN performance on distorted images with human performance. The results show that DNNs decline rapidly when images are distorted compared to humans, unless DNNs were trained for a specific kind of distortion.

\cite{slack2020foolinglime} investigate the reliability and robustness of the post-hoc explanation techniques LIME and SHAP. They devise a \emph{scaffolding} technique to attack these techniques, allowing them to hide biases of models and give arbitrary explanations.

\cite{saisub2020clusterinterpret} investigate the trade-off between accuracy and interpretability in clustering. They define a measurement for the interpretability of a cluster, by finding the feature value that most of the cluster nodes share. Thus, interpretability is defined as measurement of similarity. With this measurement, the authors show experimentally how to trade-off interpretability and clustering value in a clustering. However, there aren't any human trials to benchmark this definition of interpretability yet.

\cite{hooker2019interpretabilitymethods} detail an automated benchmark for interpretability methods for neural networks using removal of input dimensions and subsequent retraining. While this approach can measure the correlation between explained and actual importance of features, it does not take into account human perception and biases.

\cite{mohseni2018humanevaluation} propose a human-grounded benchmark for evaluating explanations. Human annotators create attention masks that are compared with saliency maps generated by XAI algorithms. A trial showed differences in human and AI explanations, for example a human focus on facial features when recognizing animals, while saliency maps had no focus on specific body parts. In addition, they showed human bias towards different explanation errors.

\cite{schneider2020deceptiveexplanations} investigate trust in explanations. They show that AI outperforms humans at detecting altered, deceptive explanations. 

\cite{10.1145/2939672.2939778} evaluate LIME with both expert and non-expert users, making them choose between different models based on explanation quality and measuring understanding of the underlying model. These experiments show the usefulness of explanations for machine learning-related tasks and show first indications of aspects of explanation quality.

\cite{kaur2020interpretinginterpretability} investigate the use of interpretability by data scientists. A survey showed that data scientists overestimate the quality of XAI methods and misinterpret the explanations provided. While this work does not focus on end users, it highlights the importance of clearly communicating the limits of XAI and investigating how explanations are used and interpreted by humans in real-life scenarios.

\section{VitrAI Prototype for XAI}
\label{sec:prototype}

 \emph{VitrAI} is a web-based service with the goal of demonstrating XAI algorithms in the context of real-life scenarios and evaluating performance and comprehensibility of XAI methods by non-specialists.

The platform consists of two core sections. The first one is the demo section, in which XAI explanations are exhibited in three scenarios. These are described in \ref{subsubsec:Demo section}. The purpose of the demo section is to provide a user-friendly introduction to XAI, requiring no setup or data input. It is intended as an interactive demo for talks, exhibitions and lectures.

The other one is the user-controlled section, where users can choose from several pre-trained machine learning models and provide own data input with subsequent explanations by XAI methods. This section allows for deeper examination and experimentation with XAI methods, for example during user testing.

For each section, different XAI methods are implemented, they are depicted in \ref{subsec:Supported XAI methods}. \emph{VitrAI's} name is a compound of the Latin word vitrum (glass) and Artificial Intelligence.

\subsection{Machine Learning Tasks and Data Types}
\label{subsec:Machine learning and data types}

Across the platform, a mix of general and domain-specific tasks and datasets is encountered. On the one hand, there is unstructured data like images and texts. On the other hand, classical tabular data is present. Each dataset belongs to a machine learning task. Presently, XAI is closely related to supervised learning tasks~\cite{10.1145/3359992.3366639}, hence all considered tasks are supervised in nature.

The following machine learning tasks are considered: supervised text classification, supervised image classification and supervised binary classification of tabular data. For such tasks, many machine learning models have been trained and deployed in the last few years, in science as well as in the industry. Hence, these tasks are suited for the evaluation of XAI methods.

\subsection{Main Sections}
\label{subsec:Main sections}


\subsubsection{Demo Section}
\label{subsubsec:Demo section}
\noindent The foundation of \emph{VitrAI's} demo functionality consists of three real-life scenarios. The first one is called \emph{Public Transport} and is about supervised text classification with five classes. It contains six user complaints about public transport in German language. These complaints are based on true events and revolve around buses leaving too late or too early, unfriendly bus drivers or service issues related to public transport. Each text or complaint is assigned a label, e.g. "ride not on time" or "wrong bus stop", by a machine learning model. The used machine learning algorithm is a Convolutional Neural Network built and trained with spaCy\footnote{\url{https://spacy.io/}} and achieves an accuracy score of 92\%.

The second scenario is called \emph{Car Brands} and is a supervised image classification task. This scenario deals with car images from various angles and distances. The seven images are gathered from the CompCars dataset\footnote{\url{http://mmlab.ie.cuhk.edu.hk/datasets/comp_cars/index.html}} and are assigned one of seven make labels, e.g. "Volkswagen" or "Skoda". The machine learning model is a fine-tuned transfer learning model and achieves an accuracy score of 87\%.

The third setting, \emph{Weather Forecast}, is a supervised binary classification task. It deals with five samples from the "Rain in Australia" dataset\footnote{\url{https://www.kaggle.com/jsphyg/weather-dataset-rattle-package}} that displays 10 years of weather observations from locations across Australia. From a data focused point of view, this data is an example of tabular data, i.e. numerical and categorical features like wind direction, wind speed in the afternoon or atmospheric pressure in the morning of the day before. The machine learning model predicts rainfall for the next day in a binary fashion ("Yes" or "No"). We used a tree-based model trained with scikit-learn\footnote{\url{https://scikit-learn.org/stable/}} that shows an accuracy score of 84\%.

Next to the three scenarios there is a \emph{Dataset Information} functionality that, based on Pandas Profiling\footnote{\url{https://github.com/pandas-profiling/pandas-profiling}}, gathers dataset facts and statistics like variable types, warnings about missing features, distributions of features and correlation matrices. With this built-in functionality, users can get an overview of tabular data before diving deeper by using XAI algorithms. Even with well-functioning XAI methods a sound understanding of the underlying data is obligatory which is why we decided to include the \emph{Dataset Information} functionality.

Note that in the demo section, the training of custom machine learning models is not possible. Instead, several pre-trained models exist for each scenario. These existing models were trained with the goal of achieving a decent accuracy score in order to subsequently obtain reasonable explanation quality. At the same time, the models should not be overly complex, so that an average specialist can build them. For example, in the text scenario the machine learning model was built and trained with spaCy in a straightforward manner and achieves an accuracy score of 92\%. These requirements make for a setting in which XAI methods are used in practice and can therefore be realistically evaluated.

An overview of the demo section is depicted in \autoref{fig:images/demosection}.

\begin{figure}[h]
 \includegraphics[width=\textwidth]{images/demosection}
 \caption{Overview of the demo section with selected \emph{Car Brands} scenario.}
 \label{fig:images/demosection}
\end{figure}

\subsubsection{User-controlled Section}
\label{subsubsec:User-controlled section}

\noindent In comparison to the demo section, in the user-controlled section users can create custom machine learning models and predict self-provided samples with subsequent explanations. Regarding training, only text models can be created. For this purpose, an earlier software development project at Fraunhofer IAO is used. Here, a custom text classification model can be trained on a user-defined dataset with common machine learning libraries. 

Concerning prediction, users can select pre-trained machine learning models (and thus, a task and data type). In the case of a text classification model, the user can enter a text sample in the user interface (see \autoref{fig:images/predict_text}). Similarly, in the case of an image classification model, a sample image can be uploaded and when choosing a binary classification model, a single-line csv file containing the features as columns should be uploaded.

\begin{figure}[h]
\includegraphics[width=\textwidth]{images/predict_text}
\caption{Users first choose from a list of pre-trained models and, in the case of a text classification model, can provide a text sample which will be classified and explained later.}
\label{fig:images/predict_text}
\end{figure}

\subsection{Supported XAI Methods}
\label{subsec:Supported XAI methods}

\noindent In total, the following four XAI methods are implemented: Layer-Wise Relevance Propagation (LRP)~\cite{Bach2015OnPE}, LIME~\cite{10.1145/2939672.2939778}, SHAP~\cite{NIPS2017_7062}, and scikit-learn's permutation importances\footnote{\url{https://scikit-learn.org/stable/modules/permutation_importance.html}}. Each scenario features at least two different XAI approaches. \autoref{tab:usedxaimethods} shows the implemented XAI approaches for each scenario.

\begin{table}
\caption{Considered scenarios with implemented XAI methods}
\label{tab:usedxaimethods}
\begin{center}
\begin{tabular}{r@{\quad}c@{\quad}c@{\quad}c@{\quad}c}
\hline\rule{0pt}{12pt}
& LRP& LIME& SHAP& Permutation Importances\\[2pt]
\hline\rule{0pt}{12pt}
Public Transport & \checkmark& \checkmark& & \\
Car Brands & & \checkmark& \checkmark& \\
Weather Forecast & & \checkmark& \checkmark& \checkmark \\
\hline
\end{tabular}
\end{center}
\end{table}

The used XAI algorithms are well-studied and established in the field of XAI~\cite{Tjoa2020ASO}. With permutation importances,  \emph{VitrAI} does not only support instance-based explanations but also an explanation approach related to whole machine learning models. It is understood that permutation importances provide a rather rudimentary form of model insight, but due to their low complexity and easy usage, they can readily be used by practitioners.

\autoref{fig:images/shap_car}, \autoref{fig:images/lime_car}, \autoref{fig:images/lime_text} and \autoref{fig:images/lrp_text} show the XAI algorithms used in the Text and Image Classification scenario.

\begin{figure}[h]
 \includegraphics[width=\textwidth]{images/shap_car}
 \caption{SHAP explanation for the true prediction (nearly 100\% class probability) of the car make \emph{BMW}.}
 \label{fig:images/shap_car}
\end{figure}

\begin{figure}[h!]
	\begin{center}
	\includegraphics[scale=0.4]{images/lime_car}
	\end{center}
	\caption{LIME explanation for the true prediction (nearly 100\% class probability) of the car make \emph{BMW}.}
	\label{fig:images/lime_car}
\end{figure}

\begin{figure}[h!]
	\begin{center}
	\includegraphics[scale=0.4]{images/lime_text}
	\end{center}
	\caption{LIME explanation for the prediction of a complaint type.}
	\label{fig:images/lime_text}
\end{figure}

\begin{figure}[h!]
 \includegraphics[width=\textwidth]{images/lrp_text}
 \caption{LRP explanation for the prediction of a complaint type.}
 \label{fig:images/lrp_text}
\end{figure}

\subsection{Architecture and Technologies Used}

\noindent  \emph{VitrAI} is built according to the microservice pattern. This means that functionally independent components are encapsulated as stand-alone applications in Docker\footnote{\url{https://www.docker.com/}} containers. The containers are orchestrated with Docker Compose. The core functionality of  \emph{VitrAI} is in the modules, which are divided according to the type of data processed. For the frontend, Angular\footnote{\url{https://angular.io/}} is used in order to apply the component libraries Nebular\footnote{\url{https://akveo.github.io/nebular/}} and PrimeNG\footnote{\url{https://www.primefaces.org/primeng/}}. For the backend, technologies used are Django\footnote{\url{https://www.djangoproject.com/}}, Flask\footnote{\url{https://flask.palletsprojects.com/en/1.1.x/}}, CouchDB\footnote{\url{https://couchdb.apache.org/}} and the machine learning libraries TensorFlow\footnote{\url{https://www.tensorflow.org/}}, Keras\footnote{\url{https://keras.io/}}, scikit-learn and spaCy. 

\section{Preliminary Findings}

\noindent To begin with, we experienced challenges regarding uniform implementation and representation of different XAI methods. Although there are frameworks that orchestrate different methods, they either lack XAI approaches, present explanations heterogeneously or show dependency shortcomings (for example, iNNvestigate~\cite{JMLR:v20:18-540} and AI Explainability 360~\cite{aix360-sept-2019} lack support of TensorFlow 2.0/tf.keras models). The last issue also applies to different XAI implementations and overall results in challenging usability (e.g. existing machine learning models may need to be retrained for compatibility reasons). 

We preliminarily evaluated the image and text classification tasks. In the \emph{Car Brands} scenario, LIME highlights background regions and therefore irrelevant regions in four out of seven samples (see \autoref{fig:images/lime_car} for an example). This applies to positive as well as negative influences. On the other hand, in every sample at least parts of the highlighted regions are plausible, e.g. the brand logo or characteristic grilles are marked as positive influences (see part of the grille in \autoref{fig:images/lime_car}). SHAP highlights incomprehensible image regions in every sample (see again \autoref{fig:images/shap_car}). In five out of seven images, at least parts of the proclaimed influences are plausible. In summary, SHAP explanations are incomprehensible more frequently than LIME's, and overall explanation quality is in need of improvement. Also, explanations match human explanations at most partly (among others, humans probably would mark both grilles in \autoref{fig:images/lime_car} as positive influences).

In the \emph{Public Transport} scenario, LRP produced helpful explanations in three out of six cases. A positive example can be seen in \autoref{fig:images/lime_text}, where "minutes", "4" and "late" are assigned positive influences for the sample's association to the class "ride not on time". Less plausible is LRP's proclamation of the word "stop" as a negative influence when dealing with the class "stop missed by bus". LIME shows slightly better explanations in as much as we find four out of six explanations to be acceptable. In for out of six cases, we assess the explanations of both algorithms as similar, while at least partial conformity with human explanation is given in three out of six samples.

Summing up for all three cases, different XAI algorithms show only partial agreement among themselves and match human expectations unsystematically. For the unsatisfactory explanations, it is not clear whether they are due to poor performance of the machine learning model, poor performance of the XAI model, or some other reason.

\section{Conclusion and Outlook}

In this paper we presented \emph{VitrAI}, a platform for demonstrating and evaluating XAI. We have outlined the main modules and XAI approaches used as well as given preliminary evaluation results, showing a need for further improvement and investigation.

In the future, we intend to add more XAI approaches, e.g. CEM Explainer~\cite{10.5555/3326943.3326998}, Lucid\footnote{\url{https://github.com/tensorflow/lucid}} or ProtoDash~\cite{DBLP:conf/icdm/GurumoorthyDCA19}. Since different methods pursue diverse explanation strategies (post-hoc/ante-hoc explanations, local/global direct explanations), for a thorough evaluation a most complete coverage of different approaches is necessary.

More importantly, we will be using  \emph{VitrAI} to further research in the area of explanation quality, as it regards end users. This evaluation shall be conducted by means of aforementioned three scenarios with involvement of humans. The focus will be on the following research questions: 

\begin{enumerate}
\item How do non-specialists assess the comprehensibility of different XAI explanations?
\item What constitutes explanation quality for end users? Can it be measured, compared and quantified?
\item To what extent do XAI explanations conform to human explanations? What are possible reasons for discrepancies?
\item How can explanation quality in XAI be improved for end users?
\end{enumerate}

An experimental evaluation with human involvement promises a better understanding of current practical shortcomings of XAI approaches and can therefore initiate future research leading to increased acceptance of XAI explanations in practice.

\subsubsection*{Acknowledgment}
This work was conducted together with students from the University of Stuttgart.

%
% ---- Bibliography ----
\bibliographystyle{bibtex/spmpsci}
\bibliography{bibliography}
\end{document}

bibliography.bib
@article{montavon2018methods,
	title = {Methods for interpreting and understanding deep neural networks},
	journal = {Digital Signal Processing},
	volume = {73},
	pages = {1 - 15},
	year = {2018},
	issn = {1051-2004},
	author = {Gregoire Montavon and Wojciech Samek and Klaus-Robert Muller},
}


@misc{doshivelez2017rigorous,
	title={Towards A Rigorous Science of Interpretable Machine Learning}, 
	author={Finale Doshi-Velez and Been Kim},
	year={2017},
	eprint={1702.08608},
	archivePrefix={arXiv},
	primaryClass={stat.ML}
}


@article{skoutas2007ontology,
  title={Ontology-based conceptual design of ETL processes for both structured and semi-structured data},
  author={Skoutas, Dimitrios and Simitsis, Alkis},
  journal={International Journal on Semantic Web and Information Systems (IJSWIS)},
  volume={3},
  number={4},
  pages={1--24},
  year={2007},
  publisher={IGI Global}
}

@inproceedings{lenz1997summarizability,
  title={Summarizability in OLAP and statistical data bases},
  author={Lenz, H-J and Shoshani, Arie},
  booktitle={Scientific and Statistical Database Management, 1997. Proceedings., Ninth International Conference on},
  pages={132--143},
  year={1997},
  organization={IEEE}
}

@incollection{wilkinson2010leveraging,
  title={Leveraging business process models for ETL design},
  author={Wilkinson, Kevin and Simitsis, Alkis and Castellanos, Malu and Dayal, Umeshwar},
  booktitle={Conceptual Modeling--ER 2010},
  pages={15--30},
  year={2010},
  publisher={Springer}
}


% Encoding: UTF-8

@Book{hoffmann2013software,
  title     = {Software-Qualit{\"a}t},
  publisher = {Springer-Verlag},
  year      = {2013},
  author    = {Hoffmann, Dirk W},
}

@Book{ludewig2013software,
  title     = {Software Engineering: Grundlagen, Menschen, Prozesse, Techniken},
  publisher = {dpunkt. verlag},
  year      = {2013},
  author    = {Ludewig, Jochen and Lichter, Horst},
}

@Book{fowler1999refactoring,
  title     = {Refactoring: improving the design of existing code},
  publisher = {Addison-Wesley Professional},
  year      = {1999},
  author    = {Fowler, Martin and Beck, Kent},
  groups    = {Code Smells},
}

@InProceedings{khomh2009exploratory,
  author       = {Khomh, Foutse and Di Penta, Massimiliano and Gueheneuc, Yann-Gael},
  title        = {An exploratory study of the impact of code smells on software change-proneness},
  booktitle    = {Reverse Engineering, 2009. WCRE'09. 16th Working Conference on},
  year         = {2009},
  pages        = {75--84},
  organization = {IEEE},
  groups       = {Code Smells},
}

@InProceedings{soh2016code,
  author       = {Soh, Z{\'e}phyrin and Yamashita, Aiko and Khomh, Foutse and Gu{\'e}h{\'e}neuc, Yann-Ga{\"e}l},
  title        = {Do Code Smells Impact the Effort of Different Maintenance Programming Activities?},
  booktitle    = {Software Analysis, Evolution, and Reengineering (SANER), 2016 IEEE 23rd International Conference on},
  year         = {2016},
  volume       = {1},
  pages        = {393--402},
  organization = {IEEE},
  groups       = {Code Smells},
}

@Article{sjoberg2013quantifying,
  author    = {Sj{\o}berg, Dag IK and Yamashita, Aiko and Anda, Bente CD and Mockus, Audris and Dyb{\aa}, Tore},
  title     = {Quantifying the effect of code smells on maintenance effort},
  journal   = {IEEE Transactions on Software Engineering},
  year      = {2013},
  volume    = {39},
  number    = {8},
  pages     = {1144--1156},
  groups    = {Code Smells},
  publisher = {IEEE},
}

@InProceedings{olbrich2009evolution,
  author       = {Olbrich, Steffen and Cruzes, Daniela S and Basili, Victor and Zazworka, Nico},
  title        = {The evolution and impact of code smells: A case study of two open source systems},
  booktitle    = {Proceedings of the 2009 3rd international symposium on empirical software engineering and measurement},
  year         = {2009},
  pages        = {390--400},
  organization = {IEEE Computer Society},
  groups       = {Code Smells},
}

@InProceedings{olbrich2010all,
  author       = {Olbrich, Steffen M and Cruzes, Daniela S and Sj{\o}berg, Dag IK},
  title        = {Are all code smells harmful? A study of God Classes and Brain Classes in the evolution of three open source systems},
  booktitle    = {Software Maintenance (ICSM), 2010 IEEE International Conference on},
  year         = {2010},
  pages        = {1--10},
  organization = {IEEE},
  groups       = {Code Smells},
}

@InProceedings{mantyla2004bad,
  author       = {Mantyla, Mika V and Vanhanen, Jari and Lassenius, Casper},
  title        = {Bad smells-humans as code critics},
  booktitle    = {Software Maintenance, 2004. Proceedings. 20th IEEE International Conference on},
  year         = {2004},
  pages        = {399--408},
  organization = {IEEE},
  groups       = {ToDo, Code Smells},
  review       = {- Einleitung enthält Informationen zu Software Qualität
- Einleitung enthält Informationen zu Metriken: OO-Metriken und Erfolge damit Qualität zu bestimmen
- Code Smells sind eher vage Anhaltspunkte und stehen damit im Kontrast zu den harten Fakten, die Metriken suggerieren -> Fowler und Beck sagen ähnliches
- subjektives Empfinden von Code Smells stimmt nicht mit den automatisiert berechneten Metriken überein und auch nicht mit denen anderer Entwickler},
}

@InProceedings{schumacher2010building,
  author       = {Schumacher, Jan and Zazworka, Nico and Shull, Forrest and Seaman, Carolyn and Shaw, Michele},
  title        = {Building empirical support for automated code smell detection},
  booktitle    = {Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement},
  year         = {2010},
  pages        = {8},
  organization = {ACM},
  groups       = {Code Smells},
  review       = {- Vergleich von automatisch und manuell gefundenen Bad Smells},
}

@Book{liggesmeyer2009software,
  title     = {Software-Qualit{\"a}t: Testen, Analysieren und Verifizieren von Software},
  publisher = {Springer Science \& Business Media},
  year      = {2009},
  author    = {Liggesmeyer, Peter},
}

@Book{kan2002metrics,
  title     = {Metrics and models in software quality engineering},
  publisher = {Addison-Wesley Longman Publishing Co., Inc.},
  year      = {2002},
  author    = {Kan, Stephen H},
}

@InProceedings{zuse1993criteria,
  author       = {Zuse, Horst},
  title        = {Criteria for program comprehension derived from software complexity metrics},
  booktitle    = {Program Comprehension, 1993. Proceedings., IEEE Second Workshop on},
  year         = {1993},
  pages        = {8--16},
  organization = {IEEE},
  groups       = {GQM},
  review       = {- Complexity und Reusability sind "similar"
- McCabe wird als Metrik für Reusability genannt
-> WMC ebenfalls Metrik für Reusability},
}

@Article{etzkorn2001automated,
  author    = {Etzkorn, Letha H. and Hughes, William E and Davis, Carl G},
  title     = {Automated reusability quality analysis of OO legacy software},
  journal   = {Information and Software Technology},
  year      = {2001},
  volume    = {43},
  number    = {5},
  pages     = {295--308},
  groups    = {GQM},
  publisher = {Elsevier},
  review    = {- mehrere Metriken werden genannt und aggregiert um Wiederverwendbarkeit vorauszusagen
- Metriken sind: LCOM, Number of Public Methods, Comment Lines (%), Commented Methods (%), Number of Comments/(NOF, NOM), WMC},
}

@Article{caldiera1994goal,
  author  = {Caldiera, VRBG and Rombach, H Dieter},
  title   = {The goal question metric approach},
  journal = {Encyclopedia of software engineering},
  year    = {1994},
  volume  = {2},
  pages   = {528--532},
  groups  = {GQM},
}

@Book{meyer1988object,
  title     = {Object-oriented software construction},
  publisher = {Prentice hall New York},
  year      = {1988},
  author    = {Meyer, Bertrand},
  volume    = {2},
  groups    = {GQM},
  review    = {- habe ich nicht gelesen, aber sagt aus, dass ein hoher Coupling Factor einen negativen Einfluss auf die Analysierbarkeit hat},
}

@Article{bandi2003predicting,
  author    = {Bandi, Rajendra K. and Vaishnavi, Vijay K. and Turk, Daniel E.},
  title     = {Predicting maintenance performance using object-oriented design complexity metrics},
  journal   = {IEEE transactions on Software Engineering},
  year      = {2003},
  volume    = {29},
  number    = {1},
  pages     = {77--87},
  publisher = {IEEE},
  review    = {- machen Aussage über Änderbarkeit und Analysierbarkeit zusammen
- ich interpretiere es als Zeichen für beides},
}

@InProceedings{ahtee2003inspections,
  author       = {Ahtee, Tero},
  title        = {Inspections and historical data in teaching software engineering project course},
  booktitle    = {Software Engineering Education and Training, 2003.(CSEE\&T 2003). Proceedings. 16th Conference on},
  year         = {2003},
  pages        = {288--297},
  organization = {IEEE},
  groups       = {Related Work},
  review       = {- aus alten Projekten werden Daten gesammelt, die den späteren Gruppen helfen sollen
- zum Beispiel: aufgetretene Probleme und Risiken, Probleme oder Empfehlungen von Werkzeugen oder Projektmanagement, Statistiken},
}

@InProceedings{wetzel2009studentische,
  author    = {Wetzel, Matthias and R{\"o}der, Holger},
  title     = {Studentische Projekte: erfolgreich oder realistisch?},
  booktitle = {SEUH},
  year      = {2009},
  pages     = {17--28},
  review    = {- nennen Kriterien für Erfolg von Studentenprojekten: 
■ Die Studierenden haben Kompetenzen im Bereich der Durchführung und des
Managements von Software-Projekten erworben.
■ Der vereinbarte Funktionsumfang des Software-Produkts ist weitgehend
vorhanden.
■ Die demonstrierte Qualität des Software-Produkts erscheint ausreichend.
■ Aufwand und Termine wurden weitgehend eingehalten},
}

@Misc{standish1995chaos,
  author = {{Standish Group International}},
  title  = {The CHAOS Report},
  year   = {1995},
  review = {Erfolgskriterien für Projekte:
■ Der Auslieferungstermin wird eingehalten.
■ Das Budget wird eingehalten.
■ Die Anforderungen an das Software-Produkt werden erfüllt.},
}

@Article{bischofs:2004:dasVirtuelleSoftwareprojekt,
  author    = {Bischofs, Ludger and Hasselbring, Wilhelm and Sauer, J{\"u}rgen and Vornberger, Oliver},
  title     = {Das Virtuelle Softwareprojekt},
  year      = {2004},
  groups    = {Prozessanalyse},
  publisher = {epos Verlag},
  review    = {praktische Vermittlung von Techniken zur systematischen Arbeit
im Team},
}

@Article{Bruegge:2015:SEP,
  author    = {Bruegge, Bernd and Krusche, Stephan and Alperowitz, Lukas},
  title     = {Software engineering project courses with industrial clients},
  journal   = {ACM Transactions on Computing Education (TOCE)},
  year      = {2015},
  volume    = {15},
  number    = {4},
  pages     = {17},
  publisher = {ACM},
  review    = {den Studenten beibringen, wie man in einem Team arbeitet und
dass sie den komplexen Softwareentwicklungsprozess verstehen},
}

@InProceedings{costa2009approach,
  author       = {Costa-Soria, Cristobal and Llavador, Manuel and del Carmen Penades, Maria},
  title        = {An approach for teaching software engineering through reverse engineering},
  booktitle    = {EAEEIE Annual Conference, 2009},
  year         = {2009},
  pages        = {1--6},
  organization = {IEEE},
  groups       = {Prozessanalyse},
  review       = {Vermittlung von reverse engineering Begründung: Studenten verstehen
schneller und leichter das System; sie können dadurch schneller
den Code modifizieren; die Studenten verbessern ihre Programmierfähigkeiten},
}

@InProceedings{Bruegge2000,
  author       = {Bruegge, Bernd and Dutoit, Allen H and Kobylinski, Rafael and Teubner, G{\"u}nter},
  title        = {Transatlantic project courses in a university environment},
  booktitle    = {Software Engineering Conference, 2000. APSEC 2000. Proceedings. Seventh Asia-Pacific},
  year         = {2000},
  pages        = {30--37},
  organization = {IEEE},
  review       = {Globalisierung -> Softwareprojekte werden immer verteilter -> Studenten
sollen Umgang mit solchen verteilten Projekten lernen (verschiedene
Kulturen, sprachen, Zeitzonen, Standards. Verwendung geeigneter
Kommunikationstools).},
}

@Article{demuth:1999:erfahrungenSoftwarepraktikum,
  author  = {Demuth, Birgit and Hu{\ss}mann, Heinrich and Zschaler, Steffen and Schmitz, Lothar},
  title   = {Erfahrungen mit einem frameworkbasierten Softwarepraktikum},
  journal = {Software Engineering im Unterricht der Hochschulen SEUH},
  year    = {1999},
  volume  = {99},
  pages   = {21--30},
  groups  = {Prozessanalyse},
  review  = {Projekt mit vorgegebenem Framework. Studenten sollen die Wiederverwendung
von OO-Software lernen und unerfahrenen OO-Programmierer
wird der Einstieg mit einem gut dokumentierten Framework erleichtert.},
}

@InProceedings{Gokhale2013,
  author       = {Gokhale, Swapna and Smith, Th{\'e}r{\`e}se and McCartney, Robert},
  title        = {Teaching software maintenance with open source software: Experiences and lessons},
  booktitle    = {2013 IEEE Frontiers in Education Conference (FIE)},
  year         = {2013},
  pages        = {1664--1670},
  organization = {IEEE},
  groups       = {Prozessanalyse},
  review       = {In der Praxis sind existierende Systeme oft schlecht geschrieben
und kaum dokumentiert. In diesen Systemen findet Wartung statt, zum
Beispiel das Beheben von Fehlern. Studenten müssen die Fähigkeiten
erlernen, solche Software effektiv und effizient zu warten.},
}

@Misc{stuprohomepage,
  author       = {{Fachbereich Informatik, Universität Stuttgart}},
  title        = {{Studienprojekte: Informationen für Studierende | Der Fachbereich Informatik | Universität Stuttgart}},
  howpublished = {\url{https://www.informatik.uni-stuttgart.de/studium/studierende/bsc-studiengaenge/softwaretechnik/studienprojekte/studi-infos-index.html}},
  note         = {[Online; URL checked on 12.07.2017]},
}

@Article{conway1968committees,
  author  = {Conway, Melvin E},
  title   = {How do committees invent},
  journal = {Datamation},
  year    = {1968},
  volume  = {14},
  number  = {4},
  pages   = {28--31},
  review  = {- Struktur einer Software ist durch Kommunikationsstrukturen der Organisation vorbestimmt},
}

@Article{hampp2006quantitative,
  author  = {Hampp, Tilmann},
  title   = {{Quantitative Analyse studentischer Projekte}},
  journal = {Softwaretechnik-Trends},
  year    = {2006},
  volume  = {26},
  number  = {1},
}

@Misc{sourcemeter,
  author       = {{FrontEndART Ltd.}},
  title        = {{SourceMeter - Free-to-use, Advanced Source Code Analysis Suite}},
  howpublished = {\url{https://www.sourcemeter.com/}}
}

@Misc{sonarqubeplugins,
  author       = {{SonarSource S.A.}},
  title        = {{Plugin Library - Plugins - Doc SonarQube}},
  howpublished = {\url{https://docs.sonarqube.org/display/PLUG/Plugin+Library}},
  note         = {[Online; abgerufen am 14.07.2017]},
}

@Misc{sonarqube,
  author       = {{SonarSource S.A.}},
  title        = {{Continuous Code Quality | SonarQube}},
  howpublished = {\url{https://www.sonarqube.org/}},
  note         = {[Online; abgerufen am 14.07.2017]},
}

@Misc{findbugs,
  author       = {{FindBugs}},
  title        = {{FindBugs\texttrademark - Find Bugs in Java Programs}},
  howpublished = {\url{http://findbugs.sourceforge.net/}},
  note         = {[Online; abgerufen am 14.07.2017]},
}

@Misc{pmd,
  author       = {{PMD}},
  title        = {{PMD}},
  howpublished = {\url{https://pmd.github.io/}},
  note         = {[Online; abgerufen am 14.07.2017]},
}

@Misc{checkstyle,
  author       = {{checkstyle}},
  title        = {{checkstyle - Checkstyle 8.0}},
  howpublished = {\url{http://checkstyle.sourceforge.net/}},
  note         = {[Online; abgerufen am 14.07.2017]},
}

@Book{general1977factors,
  title     = {Factors in software quality},
  publisher = {Information Systems Programs, General Electric Company},
  year      = {1977},
  author    = {General Electric Company and McCall, Jim A and Richards, Paul K and Walters, Gene F},
}

@InProceedings{mordal2009squale,
  author       = {Mordal-Manet, Karine and Balmas, Fran{\c{c}}oise and Denier, Simon and Ducasse, St{\'e}phane and Wertz, Harald and Laval, Jannik and Bellingard, Fabrice and Vaillergues, Philippe},
  title        = {The squale model—A practice-based industrial quality model},
  booktitle    = {Software Maintenance, 2009. ICSM 2009. IEEE International Conference on},
  year         = {2009},
  pages        = {531--534},
  organization = {IEEE},
}

@InProceedings{abreu1994object,
  author    = {Abreu, Fernando Brito and Carapu{\c{c}}a, Rog{\'e}rio},
  title     = {Object-oriented software engineering: Measuring and controlling the development process},
  booktitle = {Proceedings of the 4th international conference on software quality},
  year      = {1994},
  volume    = {186},
  pages     = {1--8},
  review    = {- aus diesem Paper kommen die MOOD Metriken},
}

@Article{chidamber1994metrics,
  author    = {Chidamber, Shyam R and Kemerer, Chris F},
  title     = {A metrics suite for object oriented design},
  journal   = {IEEE Transactions on software engineering},
  year      = {1994},
  volume    = {20},
  number    = {6},
  pages     = {476--493},
  publisher = {IEEE},
  review    = {- aus diesem Paper kommen die ck Metriken},
}

@Article{li1993object,
  author    = {Li, Wei and Henry, Sallie},
  title     = {Object-oriented metrics that predict maintainability},
  journal   = {Journal of systems and software},
  year      = {1993},
  volume    = {23},
  number    = {2},
  pages     = {111--122},
  publisher = {Elsevier},
  review    = {- Data Abstraction Coupling Metrik},
}

@InProceedings{bruntink2004predicting,
  author       = {Bruntink, Magiel and Van Deursen, Arie},
  title        = {Predicting class testability using object-oriented metrics},
  booktitle    = {Fourth IEEE International Workshop on Source Code Analysis and Manipulation},
  year         = {2004},
  pages        = {136--145},
  organization = {IEEE},
  review       = {- testability metrics},
}

@Article{mccabe1976complexity,
  author    = {McCabe, Thomas J},
  title     = {A complexity measure},
  journal   = {IEEE Transactions on software Engineering},
  year      = {1976},
  number    = {4},
  pages     = {308--320},
  publisher = {IEEE},
  review    = {- McCabe Metrik},
}

@InProceedings{wagner2012quamoco,
  author       = {Wagner, Stefan and Lochmann, Klaus and Heinemann, Lars and Kl{\"a}s, Michael and Trendowicz, Adam and Pl{\"o}sch, Reinhold and Seidl, Andreas and Goeb, Andreas and Streit, Jonathan},
  title        = {The quamoco product quality modelling and assessment approach},
  booktitle    = {Proceedings of the 34th international conference on software engineering},
  year         = {2012},
  pages        = {1133--1142},
  organization = {IEEE Press},
}

@InProceedings{deissenboeck2009software,
  author       = {Deissenboeck, Florian and Juergens, Elmar and Lochmann, Klaus and Wagner, Stefan},
  title        = {Software quality models: Purposes, usage scenarios and requirements},
  booktitle    = {WOSQ'09.},
  year         = {2009},
  pages        = {9--14},
  organization = {IEEE},
}

@Article{briand1999unified,
  author    = {Briand, Lionel C. and Daly, John W. and Wust, Jurgen K},
  title     = {A unified framework for coupling measurement in object-oriented systems},
  journal   = {IEEE Transactions on software Engineering},
  year      = {1999},
  volume    = {25},
  number    = {1},
  pages     = {91--121},
  publisher = {IEEE},
}

@Article{chidamber1998managerial,
  author    = {Chidamber, Shyam R and Darcy, David P and Kemerer, Chris F},
  title     = {Managerial use of metrics for object-oriented software: An exploratory analysis},
  journal   = {IEEE Transactions on software Engineering},
  year      = {1998},
  volume    = {24},
  number    = {8},
  pages     = {629--639},
  publisher = {IEEE},
}

@Article{harrison2000experimental,
  author    = {Harrison, Rachel and Counsell, Steve and Nithi, Reuben},
  title     = {Experimental assessment of the effect of inheritance on the maintainability of object-oriented systems},
  journal   = {Journal of Systems and Software},
  year      = {2000},
  volume    = {52},
  number    = {2},
  pages     = {173--179},
  publisher = {Elsevier},
}

@InProceedings{e1996evaluating,
  author       = {e Abreu, F Brito and Melo, Walcelio},
  title        = {Evaluating the impact of object-oriented design on software quality},
  booktitle    = {Software Metrics Symposium, 1996., Proceedings of the 3rd International},
  year         = {1996},
  pages        = {90--99},
  organization = {IEEE},
}

@TechReport{iso25010,
  author      = {{ISO 25010}},
  title       = {{Systems and software engineering -- Systems and software Quality Requirements and Evaluation (SQuaRE) -- System and software quality models}},
  institution = {International Organization for Standardization},
  year        = {2011},
  type        = {Norm},
  number      = {ISO/IEC 25010:2011},
  address     = {Genf, CH},
  month       = mar,
  key         = {ISO/IEC 25010:2011},
}

@TechReport{iso9126,
  author      = {{ISO 9126}},
  title       = {{Software engineering -- Product quality -- Part 1: Quality model}},
  institution = {International Organization for Standardization},
  year        = {2001},
  type        = {Norm},
  number      = {ISO/IEC 9126-1:2001},
  address     = {Genf, CH},
  month       = jun,
  key         = {ISO/IEC 9126-1:2001},
}

@Misc{metricsreloaded,
  author       = {{BasLeijdekkers}},
  title        = {{BasLeijdekkers/MetricsReloaded: Automated code metrics plugin for IntelliJ IDEA}},
  howpublished = {\url{https://github.com/BasLeijdekkers/MetricsReloaded}},
  note         = {[Online; abgerufen am 14.07.2017]},
}

@Misc{metrics,
  author       = {{sauerf}},
  title        = {{Metrics 1.3.6}},
  howpublished = {\url{http://metrics.sourceforge.net/}},
  note         = {[Online; abgerufen am 14.07.2017]},
}

@Misc{metricsplugin,
  author       = {Lance Walton},
  title        = {{Eclipse Metrics Plugin - State Of Flow}},
  howpublished = {\url{http://eclipse-metrics.sourceforge.net/}},
  note         = {[Online; abgerufen am 14.07.2017]},
}

@Misc{resourcestandardmetrics,
  author       = {{M Squared Technologies}},
  title        = {{Resource Standard Software Source Code Metrics For C, C++, C\# and Java}},
  howpublished = {\url{http://msquaredtechnologies.com/m2rsm/index.htm}},
  note         = {[Online; abgerufen am 14.07.2017]},
}

@Misc{jhawk,
  author       = {{Virtual Machinery}},
  title        = {{JHawk - the Java metrics tool - Product Overview}},
  howpublished = {\url{http://www.virtualmachinery.com/jhawkprod.htm}},
  note         = {[Online; abgerufen am 14.07.2017]},
}

@Misc{imagix,
  author       = {{Imagix}},
  title        = {{Static Analysis, Software Metrics and Test - Imagix}},
  howpublished = {\url{https://www.imagix.com/products/static-analysis-and-metrics.html}},
  note         = {[Online; abgerufen am 14.07.2017]},
}

@Misc{essentialmetrics,
  author       = {{Power Software}},
  title        = {{Power Software - Products - Essential Metrics}},
  howpublished = {\url{http://www.powersoftware.com/em/}},
  note         = {[Online; abgerufen am 14.07.2017]},
}

@Misc{javasourcecodemetrics,
  author       = {{Semantic Designs}},
  title        = {{Semantic Designs: Java Source Code Metrics}},
  howpublished = {\url{http://www.semanticdesigns.com/Products/Metrics/JavaMetrics.html}},
  note         = {[Online; abgerufen am 14.07.2017]},
}

@Misc{gitsvn,
  author       = {{Git}},
  title        = {{Git - git-svn Documentation}},
  howpublished = {\url{https://git-scm.com/docs/git-svn}},
  note         = {[Online; abgerufen am 13.08.2017]},
}

@InProceedings{poncin2011mining,
  author       = {Poncin, Wouter and Serebrenik, Alexander and van den Brand, Mark},
  title        = {Mining student capstone projects with FRASR and ProM},
  booktitle    = {ACM international conference companion on Object oriented programming systems languages and applications companion},
  year         = {2011},
  pages        = {87--96},
  organization = {ACM},
}

@Article{taylor1990interpretation,
  author    = {Taylor, Richard},
  title     = {Interpretation of the correlation coefficient: a basic review},
  journal   = {Journal of diagnostic medical sonography},
  year      = {1990},
  volume    = {6},
  number    = {1},
  pages     = {35--39},
  publisher = {Sage Publications Sage CA: Thousand Oaks, CA},
  review    = {- Interpretation des Korrelationskoeffizienten},
}

@InProceedings{poncin2011process,
  author       = {Poncin, Wouter and Serebrenik, Alexander and Van Den Brand, Mark},
  title        = {Process mining software repositories},
  booktitle    = {Software Maintenance and Reengineering (CSMR), 2011 15th European Conference on},
  year         = {2011},
  pages        = {5--14},
  organization = {IEEE},
}

@InProceedings{van2005prom,
  author       = {Van Dongen, Boudewijn F and de Medeiros, Ana Karla A and Verbeek, HMW and Weijters, AJMM and Van Der Aalst, Wil MP},
  title        = {The prom framework: A new era in process mining tool support.},
  booktitle    = {ICATPN},
  year         = {2005},
  volume       = {3536},
  pages        = {444--454},
  organization = {Springer},
}

@Article{pearson1895note,
  author    = {Pearson, Karl},
  title     = {Note on regression and inheritance in the case of two parents},
  journal   = {Proceedings of the Royal Society of London},
  year      = {1895},
  volume    = {58},
  pages     = {240--242},
  publisher = {JSTOR},
}

@InProceedings{ostberg2014automatically,
  author       = {Ostberg, Jan-Peter and Wagner, Stefan},
  title        = {On Automatically Collectable Metrics for Software Maintainability Evaluation},
  booktitle    = {Software Measurement and the International Conference on Software Process and Product Measurement (IWSM-MENSURA), 2014 Joint Conference of the International Workshop on},
  year         = {2014},
  pages        = {32--37},
  organization = {IEEE},
}

@Comment{jabref-meta: databaseType:bibtex;}

@Comment{jabref-meta: groupstree:
0 AllEntriesGroup:;
1 ExplicitGroup:Code Smells\;0\;;
1 ExplicitGroup:ToDo\;0\;;
1 ExplicitGroup:GQM\;0\;;
1 ExplicitGroup:Related Work\;0\;;
1 ExplicitGroup:Prozessanalyse\;0\;;
}

@article{schwaber2011scrum,
  title={The scrum guide},
  author={Schwaber, Ken and Sutherland, Jeff},
  journal={Scrum Alliance},
  volume={21},
  year={2011}
}

@inproceedings{geirhos2018humandnnbenchmark,
author = {Geirhos, Robert and Temme, Carlos R. Medina and Rauber, Jonas and Sch\"{u}tt, Heiko H. and Bethge, Matthias and Wichmann, Felix A.},
title = {Generalisation in Humans and Deep Neural Networks},
year = {2018},
publisher = {Curran Associates Inc.},
address = {Red Hook, NY, USA},
booktitle = {Proceedings of the 32nd International Conference on Neural Information Processing Systems},
pages = {7549–7561},
numpages = {13},
location = {Montreal, Canada},
series = {NIPS’18}
}

  

@inproceedings{slack2020foolinglime,
author = {Slack, Dylan and Hilgard, Sophie and Jia, Emily and Singh, Sameer and Lakkaraju, Himabindu},
title = {Fooling LIME and SHAP: Adversarial Attacks on Post Hoc Explanation Methods},
year = {2020},
isbn = {9781450371100},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
booktitle = {Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society},
pages = {180–186},
numpages = {7},
keywords = {model interpretability, bias detection, black box explanations, adversarial attacks},
location = {New York, NY, USA},
series = {AIES ’20}
}

@article{lipton2018mythosinterpretability,
author = {Lipton, Zachary C.},
title = {The Mythos of Model Interpretability},
year = {2018},
issue_date = {May-June 2018},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
volume = {16},
number = {3},
issn = {1542-7730},
journal = {Queue},
month = jun,
pages = {31–57},
numpages = {27}
}

@inproceedings{zhou2020intelligibility,
author = {Zhou, Yishan and Danks, David},
title = {Different “Intelligibility” for Different Folks},
year = {2020},
isbn = {9781450371100},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
booktitle = {Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society},
pages = {194–199},
numpages = {6},
keywords = {intelligibility, explainability, prediction algorithms},
location = {New York, NY, USA},
series = {AIES ’20}
}

@inproceedings{zhou2020assessingposthoc,
author = {Zhou, Tongyu and Sheng, Haoyu and Howley, Iris},
title = {Assessing Post-Hoc Explainability of the BKT Algorithm},
year = {2020},
isbn = {9781450371100},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
booktitle = {Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society},
pages = {407–413},
numpages = {7},
keywords = {explainable ai, post-hoc explanations, evaluation of xai systems, communicating algorithmic systems, interpretability of algorithms},
location = {New York, NY, USA},
series = {AIES ’20}
}

@inproceedings{saisub2020clusterinterpret,
author = {Saisubramanian, Sandhya and Galhotra, Sainyam and Zilberstein, Shlomo},
title = {Balancing the Tradeoff Between Clustering Value and Interpretability},
year = {2020},
isbn = {9781450371100},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
booktitle = {Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society},
pages = {351–357},
numpages = {7},
keywords = {centroid-based clustering, interpretability},
location = {New York, NY, USA},
series = {AIES ’20}
}
  
@incollection{hooker2019interpretabilitymethods,
title = {A Benchmark for Interpretability Methods in Deep Neural Networks},
author = {Hooker, Sara and Erhan, Dumitru and Kindermans, Pieter-Jan and Kim, Been},
booktitle = {Advances in Neural Information Processing Systems 32},
editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d. Alch\'{e}-Buc and E. Fox and R. Garnett},
pages = {9737--9748},
year = {2019},
publisher = {Curran Associates, Inc.}
}

@article{Bach2015OnPE,
  title={On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation},
  author={S. Bach and Alexander Binder and Gr{\'e}goire Montavon and F. Klauschen and K. M{\"u}ller and W. Samek},
  journal={PLoS ONE},
  year={2015},
  volume={10}
}

@inproceedings{10.1145/2939672.2939778,
author = {Ribeiro, Marco Tulio and Singh, Sameer and Guestrin, Carlos},
title = {"Why Should I Trust You?": Explaining the Predictions of Any Classifier},
year = {2016},
isbn = {9781450342322},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
booktitle = {Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining},
pages = {1135–1144},
numpages = {10},
keywords = {explaining machine learning, interpretability, black box classifier, interpretable machine learning},
location = {San Francisco, California, USA},
series = {KDD '16}
}

@incollection{NIPS2017_7062,
title = {A Unified Approach to Interpreting Model Predictions},
author = {Lundberg, Scott M and Lee, Su-In},
booktitle = {Advances in Neural Information Processing Systems 30},
editor = {I. Guyon and U. V. Luxburg and S. Bengio and H. Wallach and R. Fergus and S. Vishwanathan and R. Garnett},
pages = {4765--4774},
year = {2017},
publisher = {Curran Associates, Inc.}
}

@article{Tjoa2020ASO,
  title={A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI},
  author={Erico Tjoa and Cuntai Guan},
  journal={IEEE transactions on neural networks and learning systems},
  year={2020},
  volume={PP}
}

  

@inproceedings{DBLP:conf/icdm/GurumoorthyDCA19,
  author    = {Karthik S. Gurumoorthy and
               Amit Dhurandhar and
               Guillermo A. Cecchi and
               Charu C. Aggarwal},
  editor    = {Jianyong Wang and
               Kyuseok Shim and
               Xindong Wu},
  title     = {Efficient Data Representation by Selecting Prototypes with Importance
               Weights},
  booktitle = {2019 {IEEE} International Conference on Data Mining, {ICDM} 2019,
               Beijing, China, November 8-11, 2019},
  pages     = {260--269},
  publisher = {{IEEE}},
  year      = {2019},
  timestamp = {Thu, 06 Feb 2020 10:29:37 +0100},
  biburl    = {https://dblp.org/rec/conf/icdm/GurumoorthyDCA19.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}

@inproceedings{10.5555/3326943.3326998,
author = {Dhurandhar, Amit and Chen, Pin-Yu and Luss, Ronny and Tu, Chun-Chen and Ting, Paishun and Shanmugam, Karthikeyan and Das, Payel},
title = {Explanations Based on the Missing: Towards Contrastive Explanations with Pertinent Negatives},
year = {2018},
publisher = {Curran Associates Inc.},
address = {Red Hook, NY, USA},
booktitle = {Proceedings of the 32nd International Conference on Neural Information Processing Systems},
pages = {590–601},
numpages = {12},
location = {Montr\'{e}al, Canada},
series = {NIPS'18}
}

@article{JMLR:v20:18-540,
  author  = {Maximilian Alber and Sebastian Lapuschkin and Philipp Seegerer and Miriam H{{\"a}}gele and Kristof T. Sch{{\"u}}tt and Gr{{\'e}}goire Montavon and Wojciech Samek and Klaus-Robert M{{\"u}}ller and Sven D{{\"a}}hne and Pieter-Jan Kindermans},
  title   = {iNNvestigate Neural Networks!},
  journal = {Journal of Machine Learning Research},
  year    = {2019},
  volume  = {20},
  number  = {93},
  pages   = {1-8}
}


@misc{aix360-sept-2019,
title = "One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques",
author = {Vijay Arya and Rachel K. E. Bellamy and Pin-Yu Chen and Amit Dhurandhar and Michael Hind
and Samuel C. Hoffman and Stephanie Houde and Q. Vera Liao and Ronny Luss and Aleksandra Mojsilovi\'c
and Sami Mourad and Pablo Pedemonte and Ramya Raghavendra and John Richards and Prasanna Sattigeri
and Karthikeyan Shanmugam and Moninder Singh and Kush R. Varshney and Dennis Wei and Yunfeng Zhang},
year = {2019},
}

@inproceedings{mittelstadt2019explainingexplanations,
author = {Mittelstadt, Brent and Russell, Chris and Wachter, Sandra},
title = {Explaining Explanations in AI},
year = {2019},
isbn = {9781450361255},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
pages = {279–288},
numpages = {10},
keywords = {Interpretability, Accountability, Philosophy of Science, Explanations},
location = {Atlanta, GA, USA},
series = {FAT* '19}
}

@article{gunning2017explainable,
  title={Explainable Artificial Intelligence (XAI)},
  author={Gunning, David},
  publisher={Defense Advanced Research Projects Agency},
  year = {2017}
}

@article {dressel2018compas,
	author = {Dressel, Julia and Farid, Hany},
	title = {The accuracy, fairness, and limits of predicting recidivism},
	volume = {4},
	number = {1},
	elocation-id = {eaao5580},
	year = {2018},
	publisher = {American Association for the Advancement of Science},
	journal = {Science Advances}
}

@inproceedings{10.1145/3359992.3366639,
author = {Morichetta, Andrea and Casas, Pedro and Mellia, Marco},
title = {EXPLAIN-IT: Towards Explainable AI for Unsupervised Network Traffic Analysis},
year = {2019},
isbn = {9781450369992},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
booktitle = {Proceedings of the 3rd ACM CoNEXT Workshop on Big DAta, Machine Learning and Artificial Intelligence for Data Communication Networks},
pages = {22–28},
numpages = {7},
keywords = {High-Dimensional Data, Explainable AI, Network Measurements, Clustering},
location = {Orlando, FL, USA},
series = {Big-DAMA '19}
}


@article{rudin2019stopblackbox,
  title={Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead},
  author={Rudin, Cynthia},
  journal={Nature Machine Intelligence},
  volume={1},
  number={5},
  pages={206--215},
  year={2019},
  publisher={Nature Publishing Group}
}

@article{letham2015interpretable,
  title={Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model},
  author={Letham, Benjamin and Rudin, Cynthia and McCormick, Tyler H and Madigan, David and others},
  journal={The Annals of Applied Statistics},
  volume={9},
  number={3},
  pages={1350--1371},
  year={2015},
  publisher={Institute of Mathematical Statistics}
}

@INPROCEEDINGS {zhang2018interpretablenn,
author = {Q. Zhang and Y. Wu and S. Zhu},
booktitle = {2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
title = {Interpretable Convolutional Neural Networks},
year = {2018},
volume = {},
issn = {},
pages = {8827-8836},
keywords = {visualization;semantics;integrated circuits;convolutional neural networks;task analysis;training;entropy},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
month = {jun}
}

@ARTICLE{das2020xaisurvey,
       author = {{Das}, Arun and {Rad}, Paul},
        title = "{Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey}",
      journal = {arXiv e-prints},
     keywords = {Computer Science - Computer Vision and Pattern Recognition, Computer Science - Artificial Intelligence, Computer Science - Machine Learning},
         year = 2020,
        month = jun,
          eid = {arXiv:2006.11371},
        pages = {arXiv:2006.11371},
archivePrefix = {arXiv},
       eprint = {2006.11371},
 primaryClass = {cs.CV},
       adsurl = {https://ui.adsabs.harvard.edu/abs/2020arXiv200611371D},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

@ARTICLE{mohseni2018humanevaluation,
       author = {{Mohseni}, Sina and {Block}, Jeremy E. and {Ragan}, Eric D.},
        title = "{A Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learning}",
      journal = {arXiv e-prints},
     keywords = {Computer Science - Human-Computer Interaction, Computer Science - Artificial Intelligence},
         year = 2018,
        month = jan,
          eid = {arXiv:1801.05075},
        pages = {arXiv:1801.05075},
archivePrefix = {arXiv},
       eprint = {1801.05075},
 primaryClass = {cs.HC},
       adsurl = {https://ui.adsabs.harvard.edu/abs/2018arXiv180105075M},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

@ARTICLE{schneider2020deceptiveexplanations,
       author = {{Schneider}, Johannes and {Handali}, Joshua and {Vlachos}, Michalis and {Meske}, Christian},
        title = "{Deceptive AI Explanations: Creation and Detection}",
      journal = {arXiv e-prints},
     keywords = {Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Statistics - Machine Learning},
         year = 2020,
        month = jan,
          eid = {arXiv:2001.07641},
        pages = {arXiv:2001.07641},
archivePrefix = {arXiv},
       eprint = {2001.07641},
 primaryClass = {cs.LG},
       adsurl = {https://ui.adsabs.harvard.edu/abs/2020arXiv200107641S},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

@inbook{kaur2020interpretinginterpretability,
author = {Kaur, Harmanpreet and Nori, Harsha and Jenkins, Samuel and Caruana, Rich and Wallach, Hanna and Wortman Vaughan, Jennifer},
title = {Interpreting Interpretability: Understanding Data Scientists' Use of Interpretability Tools for Machine Learning},
year = {2020},
isbn = {9781450367080},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
booktitle = {Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems},
pages = {1–14},
numpages = {14}
}
paper.bbl
\begin{thebibliography}{10}
\providecommand{\url}[1]{{#1}}
\providecommand{\urlprefix}{URL }
\expandafter\ifx\csname urlstyle\endcsname\relax
  \providecommand{\doi}[1]{DOI~\discretionary{}{}{}#1}\else
  \providecommand{\doi}{DOI~\discretionary{}{}{}\begingroup
  \urlstyle{rm}\Url}\fi

\bibitem{JMLR:v20:18-540}
Alber, M., Lapuschkin, S., Seegerer, P., H{{\"a}}gele, M., Sch{{\"u}}tt, K.T.,
  Montavon, G., Samek, W., M{{\"u}}ller, K.R., D{{\"a}}hne, S., Kindermans,
  P.J.: innvestigate neural networks!
\newblock Journal of Machine Learning Research \textbf{20}(93), 1--8 (2019)

\bibitem{aix360-sept-2019}
Arya, V., Bellamy, R.K.E., Chen, P.Y., Dhurandhar, A., Hind, M., Hoffman, S.C.,
  Houde, S., Liao, Q.V., Luss, R., Mojsilovi\'c, A., Mourad, S., Pedemonte, P.,
  Raghavendra, R., Richards, J., Sattigeri, P., Shanmugam, K., Singh, M.,
  Varshney, K.R., Wei, D., Zhang, Y.: One explanation does not fit all: A
  toolkit and taxonomy of ai explainability techniques (2019)

\bibitem{Bach2015OnPE}
Bach, S., Binder, A., Montavon, G., Klauschen, F., M{\"u}ller, K., Samek, W.:
  On pixel-wise explanations for non-linear classifier decisions by layer-wise
  relevance propagation.
\newblock PLoS ONE \textbf{10} (2015)

\bibitem{das2020xaisurvey}
{Das}, A., {Rad}, P.: {Opportunities and Challenges in Explainable Artificial
  Intelligence (XAI): A Survey}.
\newblock arXiv e-prints arXiv:2006.11371 (2020)

\bibitem{10.5555/3326943.3326998}
Dhurandhar, A., Chen, P.Y., Luss, R., Tu, C.C., Ting, P., Shanmugam, K., Das,
  P.: Explanations based on the missing: Towards contrastive explanations with
  pertinent negatives.
\newblock In: Proceedings of the 32nd International Conference on Neural
  Information Processing Systems, NIPS'18, p. 590–601. Curran Associates
  Inc., Red Hook, NY, USA (2018)

\bibitem{doshivelez2017rigorous}
Doshi-Velez, F., Kim, B.: Towards a rigorous science of interpretable machine
  learning (2017)

\bibitem{dressel2018compas}
Dressel, J., Farid, H.: The accuracy, fairness, and limits of predicting
  recidivism.
\newblock Science Advances \textbf{4}(1) (2018)

\bibitem{geirhos2018humandnnbenchmark}
Geirhos, R., Temme, C.R.M., Rauber, J., Sch\"{u}tt, H.H., Bethge, M., Wichmann,
  F.A.: Generalisation in humans and deep neural networks.
\newblock In: Proceedings of the 32nd International Conference on Neural
  Information Processing Systems, NIPS’18, p. 7549–7561. Curran Associates
  Inc., Red Hook, NY, USA (2018)

\bibitem{gunning2017explainable}
Gunning, D.: Explainable artificial intelligence (xai)  (2017)

\bibitem{DBLP:conf/icdm/GurumoorthyDCA19}
Gurumoorthy, K.S., Dhurandhar, A., Cecchi, G.A., Aggarwal, C.C.: Efficient data
  representation by selecting prototypes with importance weights.
\newblock In: J.~Wang, K.~Shim, X.~Wu (eds.) 2019 {IEEE} International
  Conference on Data Mining, {ICDM} 2019, Beijing, China, November 8-11, 2019,
  pp. 260--269. {IEEE} (2019)

\bibitem{hooker2019interpretabilitymethods}
Hooker, S., Erhan, D., Kindermans, P.J., Kim, B.: A benchmark for
  interpretability methods in deep neural networks.
\newblock In: H.~Wallach, H.~Larochelle, A.~Beygelzimer, F.~d.~Alch\'{e}-Buc,
  E.~Fox, R.~Garnett (eds.) Advances in Neural Information Processing Systems
  32, pp. 9737--9748. Curran Associates, Inc. (2019)

\bibitem{kaur2020interpretinginterpretability}
Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., Wortman~Vaughan, J.:
  Interpreting Interpretability: Understanding Data Scientists' Use of
  Interpretability Tools for Machine Learning, p. 1–14.
\newblock Association for Computing Machinery, New York, NY, USA (2020)

\bibitem{letham2015interpretable}
Letham, B., Rudin, C., McCormick, T.H., Madigan, D., et~al.: Interpretable
  classifiers using rules and bayesian analysis: Building a better stroke
  prediction model.
\newblock The Annals of Applied Statistics \textbf{9}(3), 1350--1371 (2015)

\bibitem{lipton2018mythosinterpretability}
Lipton, Z.C.: The mythos of model interpretability.
\newblock Queue \textbf{16}(3), 31–57 (2018)

\bibitem{NIPS2017_7062}
Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model
  predictions.
\newblock In: I.~Guyon, U.V. Luxburg, S.~Bengio, H.~Wallach, R.~Fergus,
  S.~Vishwanathan, R.~Garnett (eds.) Advances in Neural Information Processing
  Systems 30, pp. 4765--4774. Curran Associates, Inc. (2017)

\bibitem{mittelstadt2019explainingexplanations}
Mittelstadt, B., Russell, C., Wachter, S.: Explaining explanations in ai.
\newblock FAT* '19, p. 279–288. Association for Computing Machinery, New
  York, NY, USA (2019)

\bibitem{mohseni2018humanevaluation}
{Mohseni}, S., {Block}, J.E., {Ragan}, E.D.: {A Human-Grounded Evaluation
  Benchmark for Local Explanations of Machine Learning}.
\newblock arXiv e-prints arXiv:1801.05075 (2018)

\bibitem{montavon2018methods}
Montavon, G., Samek, W., Muller, K.R.: Methods for interpreting and
  understanding deep neural networks.
\newblock Digital Signal Processing \textbf{73}, 1 -- 15 (2018)

\bibitem{10.1145/3359992.3366639}
Morichetta, A., Casas, P., Mellia, M.: Explain-it: Towards explainable ai for
  unsupervised network traffic analysis.
\newblock In: Proceedings of the 3rd ACM CoNEXT Workshop on Big DAta, Machine
  Learning and Artificial Intelligence for Data Communication Networks,
  Big-DAMA '19, p. 22–28. Association for Computing Machinery, New York, NY,
  USA (2019)

\bibitem{10.1145/2939672.2939778}
Ribeiro, M.T., Singh, S., Guestrin, C.: "why should i trust you?": Explaining
  the predictions of any classifier.
\newblock In: Proceedings of the 22nd ACM SIGKDD International Conference on
  Knowledge Discovery and Data Mining, KDD '16, p. 1135–1144. Association for
  Computing Machinery, New York, NY, USA (2016)

\bibitem{rudin2019stopblackbox}
Rudin, C.: Stop explaining black box machine learning models for high stakes
  decisions and use interpretable models instead.
\newblock Nature Machine Intelligence \textbf{1}(5), 206--215 (2019)

\bibitem{saisub2020clusterinterpret}
Saisubramanian, S., Galhotra, S., Zilberstein, S.: Balancing the tradeoff
  between clustering value and interpretability.
\newblock In: Proceedings of the AAAI/ACM Conference on AI, Ethics, and
  Society, AIES ’20, p. 351–357. Association for Computing Machinery, New
  York, NY, USA (2020)

\bibitem{schneider2020deceptiveexplanations}
{Schneider}, J., {Handali}, J., {Vlachos}, M., {Meske}, C.: {Deceptive AI
  Explanations: Creation and Detection}.
\newblock arXiv e-prints arXiv:2001.07641 (2020)

\bibitem{slack2020foolinglime}
Slack, D., Hilgard, S., Jia, E., Singh, S., Lakkaraju, H.: Fooling lime and
  shap: Adversarial attacks on post hoc explanation methods.
\newblock In: Proceedings of the AAAI/ACM Conference on AI, Ethics, and
  Society, AIES ’20, p. 180–186. Association for Computing Machinery, New
  York, NY, USA (2020)

\bibitem{Tjoa2020ASO}
Tjoa, E., Guan, C.: A survey on explainable artificial intelligence (xai):
  Towards medical xai.
\newblock IEEE transactions on neural networks and learning systems \textbf{PP}
  (2020)

\bibitem{zhang2018interpretablenn}
Zhang, Q., Wu, Y., Zhu, S.: Interpretable convolutional neural networks.
\newblock In: 2018 IEEE/CVF Conference on Computer Vision and Pattern
  Recognition (CVPR), pp. 8827--8836. IEEE Computer Society, Los Alamitos, CA,
  USA (2018)

\bibitem{zhou2020assessingposthoc}
Zhou, T., Sheng, H., Howley, I.: Assessing post-hoc explainability of the bkt
  algorithm.
\newblock In: Proceedings of the AAAI/ACM Conference on AI, Ethics, and
  Society, AIES ’20, p. 407–413. Association for Computing Machinery, New
  York, NY, USA (2020)

\bibitem{zhou2020intelligibility}
Zhou, Y., Danks, D.: Different “intelligibility” for different folks.
\newblock In: Proceedings of the AAAI/ACM Conference on AI, Ethics, and
  Society, AIES ’20, p. 194–199. Association for Computing Machinery, New
  York, NY, USA (2020)

\end{thebibliography}
