An Overview of Recommender Systems and Machine Learning in Feature Modeling and Configuration
Alexander Felfernig
Viet-Man Le
Andrei Popescu
Mathias Uta
Thi Ngoc Trang Tran
Müslüm Atas
Recommender systems support decisions in various domains ranging from simple items such as books and movies to more complex items such as financial services, telecommunication equipment, and software systems. In this context, recommendations are determined, for example, on the basis of analyzing the preferences of similar users. In contrast to simple items which can be enumerated in an item catalog, complex items have to be represented on the basis of variability models (e.g., feature models) since a complete enumeration of all possible configurations is infeasible and would trigger significant performance issues. In this paper, we give an overview of a potential new line of research which is related to the application of recommender systems and machine learning techniques in feature modeling and configuration. In this context, we give examples of the application of recommender systems and machine learning and discuss future research issues.
main.tex
%%
%% This is file `sample-sigconf.tex',
%% generated with the docstrip utility.
%%
%% The original source files were:
%%
%% samples.dtx  (with options: `sigconf')
%% 
%% IMPORTANT NOTICE:
%% 
%% For the copyright see the source file.
%% 
%% Any modified versions of this file must be renamed
%% with new filenames distinct from sample-sigconf.tex.
%% 
%% For distribution of the original source see the terms
%% for copying and modification in the file samples.dtx.
%% 
%% This generated file may be distributed as long as the
%% original source files, as listed above, are part of the
%% same distribution. (The sources need not necessarily be
%% in the same archive or directory.)
%%
%% The first command in your LaTeX source must be the \documentclass command.
\documentclass[sigconf,twocolumn]{acmart}
%% NOTE that a single column version may be required for 
%% submission and peer review. This can be done by changing
%% the \doucmentclass[...]{acmart} in this template to 
%% \documentclass[manuscript,screen]{acmart}
%% 
%% To ensure 100% compatibility, please check the white list of
%% approved LaTeX packages to be used with the Master paper Template at
%% https://www.acm.org/publications/taps/whitelist-of-latex-packages 
%% before creating your document. The white list page provides 
%% information on how to submit additional LaTeX packages for 
%% review and adoption.
%% Fonts used in the template cannot be substituted; margin 
%% adjustments are not allowed.
%%
%%
%% \BibTeX command to typeset BibTeX logo in the docs
\AtBeginDocument{%
  \providecommand\BibTeX{{%
    \normalfont B\kern-0.5em{\scshape i\kern-0.25em b}\kern-0.8em\TeX}}}

%% Rights management information.  This information is sent to you
%% when you complete the rights form.  These commands have SAMPLE
%% values in them; it is your responsibility as an author to replace
%% the commands and values with those provided to you when you
%% complete the rights form.
\copyrightyear{2021}
\acmYear{2021}
\setcopyright{acmcopyright}\acmConference[VaMoS'21]{15th International Working Conference on Variability Modelling of Software-Intensive Systems}{February 9--11, 2021}{Krems, Austria}
\acmBooktitle{15th International Working Conference on Variability Modelling of Software-Intensive Systems (VaMoS'21), February 9--11, 2021, Krems, Austria}
\acmPrice{15.00}
\acmDOI{10.1145/3442391.3442408}
\acmISBN{978-1-4503-8824-5/21/02}

%%
%% Submission ID.
%% Use this when submitting an paper to a sponsored event. You'll
%% receive a unique submission ID from the organizers
%% of the event, and this ID should be used as the parameter to this command.
%%\acmSubmissionID{123-A56-BU3}

%%
%% The majority of ACM publications use numbered citations and
%% references.  The command \citestyle{authoryear} switches to the
%% "author year" style.
%%
%% If you are preparing content for an event
%% sponsored by ACM SIGGRAPH, you must use the "author year" style of
%% citations and references.
%% Uncommenting
%% the next command will enable that style.
%%\citestyle{acmauthoryear}

%%
%% end of the preamble, start of the body of the document source.
\begin{document}

%%
%% The "title" command has an optional parameter,
%% allowing the author to define a "short title" to be used in page headers.
\title{An Overview of Recommender Systems and Machine Learning
in Feature Modeling and Configuration}

%%
%% The "author" command and its associated commands are used to define
%% the authors and their affiliations.
%% Of note is the shared affiliation of the first two authors, and the
%% "authornote" and "authornotemark" commands
%% used to denote shared contribution to the research.


\author{Alexander Felfernig}
\affiliation{
  \institution{Institute of Software Technology, Graz University of Technology}
  \city{Graz}
  \country{Austria}}
\email{alexander.felfernig@ist.tugraz.at}




\author{Viet-Man Le}
\affiliation{
  \institution{Institute of Software Technology, Graz University of Technology}
  \city{Graz}
  \country{Austria}}
  \email{vietman.le@ist.tugraz.at}




\author{Andrei Popescu}
\affiliation{
  \institution{Institute of Software Technology, Graz University of Technology}
  \city{Graz}
  \country{Austria}}
  \email{andrei.popescu@ist.tugraz.at}



\author{Mathias Uta}
\affiliation{
  \institution{Siemens  Gas  \&  Power}
  \city{Erlangen}
  \country{Germany}}
  \email{mathias.uta@siemens.com}



\author{Thi Ngoc Trang Tran}
\affiliation{
  \institution{Institute of Software Technology, Graz University of Technology}
  \city{Graz}
  \country{Austria}}
  \email{ttrang@ist.tugraz.at}






\author{Müslüm Atas}
\affiliation{
  \institution{Institute of Software Technology, Graz University of Technology}
  \city{Graz}
  \country{Austria}}
  \email{muatas@ist.tugraz.at}



%%
%% By default, the full list of authors will be used in the page
%% headers. Often, this list is too long, and will overlap
%% other information printed in the page headers. This command allows
%% the author to define a more concise list
%% of authors' names for this purpose.
%%\renewcommand{\shortauthors}{Felfernig et al.}

%%
%% The abstract is a short summary of the work to be presented in the
%% paper.


\begin{abstract}
Recommender systems support decisions in various domains ranging from simple items such as books and movies to more complex items such as financial services, telecommunication equipment, and software systems. In this context, recommendations are determined, for example, on the basis of analyzing the preferences of similar users. In contrast to simple items which can be enumerated in an item catalog, complex items have to be represented on the basis of variability models (e.g., feature models) since a complete enumeration of all possible configurations is infeasible and would trigger significant performance issues. In this paper, we give an overview of a potential new line of research which is related to the application of recommender systems and machine learning techniques in feature modeling and configuration. In this context, we give  examples  of the application of recommender systems and machine learning and discuss  future research issues.
\end{abstract}

\maketitle

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}\label{introduction}

Feature models can be regarded as a central element of feature-oriented software development (FOSD) processes \cite{Apel2009}. Feature models can be used to represent variability and commonality properties of software artifacts and various other types of products and services  \cite{Acher2018,BeSeRu2010,Felfernigetal2014,Kang1990}. Applications thereof support users in deciding about which features should be included in a specific configuration. Feature models and variability models in general can become quite complex which makes it challenging to develop these models as well as to interact with the corresponding decision support systems \cite{Falkner2011,Pereira2018,RodasSilva2019}. In this paper, we give an overview of a potentially new research line which is related to the application of recommender systems and machine learning techniques in feature modeling and configuration scenarios.


Recommender systems can be defined \emph{as any system that guides a user in a personalized way to interesting or useful objects in a large space of possible options or that produces such objects as output} \cite{FelfernigBurke2008}. These systems use basic machine learning techniques (classification as well as prediction techniques) for being able to identify items of relevance for a user. Typical applications of recommender systems rely on a dataset that serves as an input for learning algorithms. These algorithms infer models that predict item preferences of users. Recommender system applications are manifold and range from simple items such as news  \cite{Konstan1997} to more complex items such as financial services \cite{Felfernig2006} and software systems \cite{RodasSilva2019}. 

In general, recommender systems help to infer user interests on the basis of preference definition histories, i.e., which items were preferred by users in the past. \emph{Collaborative filtering} recommender systems \cite{Konstan1997} are based on the idea of word-of-mouth promotion where items regarded as relevant by users with similar preferences (so-called \emph{nearest neighbors}) are recommended to the current user. A model-based variant thereof is \emph{matrix factorization} \cite{Koren2009} which describes the relationship between users and items on the basis of a set of hidden aspects.\footnote{In matrix factorization \cite{Koren2009}, these aspects are also denoted as \emph{features}.} \emph{Content-based filtering} \cite{Pazzani1997} is based on the idea of recommending items to the current user which are similar to those the user has liked in the past. \emph{Knowledge-based recommender systems} \cite{Burke2000} are based on explicit recommendation knowledge in terms of attributes and constraints or similarity metrics which describe the relationship between a set of customer requirements and corresponding items. Finally, \emph{group recommender systems} \cite{Felfernigetal2018} focus on the recommendation of items to groups of users instead of single users.



The major goal of this paper is to analyze recommendation scenarios in the context of feature model development and configuration. Feature models describe variability and commonality properties of items. In many cases, feature models are the basis of configurator applications associated with a potentially large user base. In application scenarios where user communities are interacting with configurators (derived from feature models), data can be collected from user interactions and exploited  to predict  user-individual preferences.

Summarizing, the contributions of this paper are the following:
\begin{itemize}
\item{We provide an overview of  recommendation and machine learning approaches in feature modeling and configuration. In this context, we focus on the basic scenarios of \emph{interactive configuration}, \emph{reconfiguration}, and \emph{feature modeling processes}.}
\item{On the basis of examples, we sketch application scenarios of recommendation technologies that open a new line of research in feature modeling and configuration.} 
\item{Finally, we discuss open research issues to be solved to further advance the related state of the art.}
\end{itemize}



The remainder of this paper is organized as follows. In Section \ref{featuremodelscsp}, we introduce an example feature model from the domain of \emph{survey software services}. In Section \ref{configurationranking}, we introduce a CSP-based representation of the example feature model. In Section \ref{configuration}, we discuss scenarios in which recommendation and related machine learning techniques can be applied to support \emph{interactive configuration}. Section \ref{reconfiguration} focuses on recommender systems and machine learning in the context of \emph{reconfiguration}. Finally, Section \ref{modeling} provides an overview of recommendation concepts that can be applied to support feature model \emph{knowledge acquisition}. Sections \ref{configuration}--\ref{modeling} also include a topic-specific discussion of open research issues. In Section \ref{conclusions}, the paper is concluded with a discussion of further research topics.



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{An Example Feature Model}\label{featuremodelscsp}

Features can be organized in a hierarchical fashion \cite{BeSeRu2010} using relationships such as \emph{mandatory} (if a parent feature is included in a configuration, the child feature must be included as well, and vice versa), \emph{optional} (given the inclusion of a parent feature, the inclusion of the corresponding child feature is optional), \emph{alternative} (if the parent feature is included, exactly one of the child features has to be included), and \emph{or} (if the parent feature is included, at least one of the child features has to be included). Furthermore, cross-tree constraints can be used to define relationships between features that do not follow the hierarchical structure of the feature model. First, \emph{excludes(a,b)} constraints prohibit the inclusion of both features ($a$ and $b$) in the same configuration. Second, \emph{requires(a,b)} constraints necessitate that if feature $a$ is included in a configuration, feature $b$ must be included as well.

An example feature model is depicted in Figure \ref{Figure1}. In this  model, \emph{license} is used to describe the selected license model where two different models are available: an \emph{advanced license} allows to include all features provided in the feature model whereas a \emph{basic license} restricts the set of selectable features. In Figure \ref{Figure1}, \emph{license} and \emph{QA} are designed as mandatory features, i.e., must be included in every survey software configuration. If a user selects \emph{ABtesting} to be included in the configuration, this also requires the inclusion of the \emph{statistics} feature. The \emph{QA} feature supports both, \emph{basic QA} and \emph{multimedia QA} questions -- at least one of these has to be included in each configuration.

\begin{figure}[ht] 
			\centering
			\fbox{
				\includegraphics[width=0.45 \textwidth]{example_fm.png}
			}
			\caption{An example feature model (\emph{survey software}).} \label{Figure1}
		\end{figure}



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Configuration Task and Solution Ranking}\label{configurationranking}	

To enable reasoning about potential solutions (configurations), a feature model has to be translated into a formal representation. One option for a formal representation of feature models are constraint satisfaction problems (CSPs) \cite{Tsang1993}. On the level of a CSP, a feature model can be defined as a \emph{configuration task} (see Definition \ref{def1}).

\definition{Configuration Task\label{def1}}. A configuration task derived from a feature model can be defined as a constraint satisfaction problem (CSP) $(V,D,C)$ where $V=\{v_1,v_2,..,v_n\}$ is a set of Boolean variables, $D=\{dom(v_1),$ $dom(v_2),..,dom(v_n)\}$ is the  set of variable domains, and $C=C_R \cup C_F$ where $C_R=\{c_1,c_2,..,c_k\}$ is a set of customer requirements (i.e., preferred inclusions and exclusions of features), and $C_F=\{c_{k+1},c_{k+2},..,c_q\}$ is a corresponding set of constraints derived from the feature model. 

In the case of basic feature models (without further attributes), $dom(v_i) = \{0,1\}$. Constraints $c_j$ can be directly derived from a feature model and represent (1) \emph{structural relationships} (e.g., a mandatory relationship) and (2) \emph {cross-tree relationships} (e.g., a \emph{requires} relationship). A set of rules of how to formalize the relationships of a feature model in terms of a set of corresponding constraints is discussed, for example, in Benavides et al. \cite{BeSeRu2010}.

On the basis of the definition of a \emph{configuration task}, we now introduce the definition of a \emph{configuration} (Definition \ref{def2}).

\definition{Configuration\label{def2}}. A configuration (solution) for a configuration task $(V,D,C)$ is an assignment $A$ of the variables in $V$ which fulfils the criteria that all constraints in $C$ are consistent with the variable assignments in $A$.

Following Definition \ref{def1}, Table \ref{tab:variablesanddomains} represents the set of CSP variables (i.e., features represented by variables in $V$) and corresponding Boolean domains ($D$) derived from the feature model in Figure \ref{Figure1}. 
	
\begin{table}[ht]
\centering 
\caption{Features (including abbreviations of feature names) and corresponding domain definitions ($1=true$, $0=false$).}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
\hline
  featurename    & abbreviation                &  domain      
\tabularnewline
\hline \hline
$survey$ &$sur$                      &   $\{0,1\}$             
\tabularnewline
\hline
$license$ & $lic$                      &   $\{0,1\}$             
\tabularnewline
\hline
$advancedlicense$ & $adlic$               &   $\{0,1\}$                 
\tabularnewline
\hline
$basiclicense$ & $baslic$           &   $\{0,1\}$                
\tabularnewline
\hline
$ABtesting$ & $AB$                &   $\{0,1\}$                
\tabularnewline
\hline
$statistics$ & $stat$                &   $\{0,1\}$                
\tabularnewline
\hline
$QA$ & $QA$                &   $\{0,1\}$                
\tabularnewline
\hline
$basicQA$ & $basQA$       &   $\{0,1\}$              
\tabularnewline
\hline
$multimediaQA$ & $mmQA$            &   $\{0,1\}$               
\tabularnewline
\hline
\end{tabular} 
\label{tab:variablesanddomains}
\end{table}


In our example, Table \ref{tab:constraintscsp} shows the constraints $C_F$ derived from the feature model depicted in Figure \ref{Figure1}. 

\begin{table}[ht]
\centering 
\caption{Constraints $C_F=\{c_1..c_9\}$ derived from Figure \ref{Figure1}.}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
\hline
  constraint                 &  CSP representation       
\tabularnewline
\hline \hline
$c_0$                      &   $survey=1$             
\tabularnewline
\hline
$c_1$                      &   $survey$ $\leftrightarrow license$             
\tabularnewline
\hline
$c_2$                    &   $ABtesting \rightarrow survey$             
\tabularnewline
\hline
$c_3$                  &   $statistics \rightarrow survey$            
\tabularnewline
\hline
$c_4$                  &   $survey \leftrightarrow QA$            
\tabularnewline
\hline
$c_5$                  &   $QA \rightarrow basicQA \lor multimediaQA$            
\tabularnewline
\hline
$c_6$           &   $(advancedlicense \leftrightarrow \neg basiclicense \land license) \land $ \tabularnewline  
                    &   $(basiclicense \leftrightarrow \neg advancedlicense \land license)$  
\tabularnewline
\hline
$c_7$            &   $\neg(ABtesting \land basiclicense)$            
\tabularnewline
\hline
$c_8$            &   $ABtesting \rightarrow statistics$            
\tabularnewline
\hline
$c_9$            &   $\neg(basiclicense \land multimediaQA)$            
\tabularnewline
\hline
\end{tabular} 
\label{tab:constraintscsp} 
\end{table}


In Table \ref{tab:constraintscsp}, $c_0$ represents a so-called \emph{root constraint} which is used to assure that (irrelevant) empty configurations are avoided. Furthermore, $c_1$ and $c_4$ represent mandatory relationships: both \emph{payment} and \emph{QA} are mandatory, i.e., every configuration has to include these features. Both of the features \emph{ABtesting} and \emph{statistics} are regarded as optional (constraints $c_2$ and $c_3$), i.e., they do not have to be part of every configuration. The features \emph{basiclicense} and \emph{advancedlicense} are regarded as part of an alternative relationship ($c_6$). Furthermore, \emph{basicQA} and \emph{multimediaQA} are part of an optional relationship ($c_5$). Finally, the features \emph{ABtesting} and \emph{basiclicense} are regarded as incompatible ($c_7$), the inclusion of feature \emph{ABtesting} requires the inclusion of feature \emph{statistics} ($c_8$), and the features \emph{basiclicense} and \emph{$multimediaQA$} are incompatible.

 Assuming the existence of the customer (user) requirement $C_R=\{c_{10}: ABtesting=1\}$, we are able to derive the configurations $A_i$ as depicted in Table \ref{tab:configurationexample}. All three configurations could be regarded as \emph{recommendation candidates}, however, we are primarily interested in solutions which are the most relevant ones for a user. In the following, we assume the existence of \emph{two example users}, namely $u_a$ and $u_b$ (both would like to have included the feature $ABtesting$).  Following the concepts of utility-based ranking which is a major element of knowledge-based recommendation \cite{Felfernig2006}, \emph{interest dimensions} can be regarded as explicit global solution properties. We denote a specific interest dimension-related \emph{user preference} of user $u_i$ with regard to dimension $d_j$ as $up(u_i,d_j)$, for example, $up(u_a,simplicity)=0.8$, $up(u_a,productivity)=0.2)$, $up(u_b,simplicity)=0.2)$, and $up(u_b,productivity)=0.8)$.
 
  \begin{table}[ht]
\centering 
\caption{Example configurations $A_1..A_3$ consistent with $C = C_F \cup C_R$ ($C_R=\{c_{10}: ABtesting=1\}$).}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
\hline
  feature                 &  $A_1$            & $A_2$         & $A_3$ 
\tabularnewline
\hline \hline
$survey$                      &   1            &  1         &  1
\tabularnewline
\hline
$advancedlicense$               &   1            &  1         &  1
\tabularnewline
\hline
$basicicense$           &   0           &  0        &  0
\tabularnewline
\hline
$ABtesting$              &   1            &  1         &  1
\tabularnewline
\hline
$statistics$          &   1            &  1         &  1
\tabularnewline
\hline
$basicQA$         &   1            &  0        &  1
\tabularnewline
\hline
$multimediaQA$            &   0           &  1         &  1
\tabularnewline
\hline
\end{tabular} 
\label{tab:configurationexample} 
\end{table}
 

The higher the value of an interest dimension (between $0..1$), the higher the corresponding interest of the user. For example, if a user is interested in \emph{simplicity}, he or she will prefer configurations with a lower number of features (lower overhead in understanding the provided software). In order to be able to rank configurations, we also need to evaluate the solution (configuration) attributes with regard to the interest dimensions \emph{simplicity} and \emph{productivity}. In this context, we omit the features \emph{survey}, \emph{license}, and \emph{QA} which are included in every configuration.
 




\begin{table}[ht]
\centering 
\caption{Utility evaluation $u$ of features $f_i$  with regard to interest dimensions $d_j$ ($u(f_i,d_j)$), for example, $u(advancedlicense,simplicity)=0.1$.}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
\hline
  features $f_i$           &  simplicity      & productivity
\tabularnewline
\hline \hline
$advancedlicense=1$                   &   0.1           & 1.0
\tabularnewline
\hline
$basiclicense=1$                 &   1.0          &  0.1   
\tabularnewline
\hline
$ABtesting=1$                 &   0.3           &  1.0
\tabularnewline
\hline
$statistics=1$                  &   0.5          &  1.0
\tabularnewline
\hline
$multimediaQA=1$         &   0.3          &  1.0
\tabularnewline
\hline
$basicQA=1$           &   1.0          &  1.0
\tabularnewline
\hline
\end{tabular} 
\label{tab:interestdimensionsattributes}
\end{table}



On the basis of the preferences of $u_a$ and $u_b$, and the information provided in Table \ref{tab:interestdimensionsattributes}, we are able to calculate the overall utility of the individual solutions $A_1..A_3$ using Formula \ref{eq:maututility}. The user-individual utilities are shown in Table \ref{tab:utilitycalculations}. In our simplified example, alternative $A_3$ is assumed to be the preferred configuration of both users. Different users could have different preferences and could also receive different recommendations that depend on their personal preferences regarding a set of interest dimensions.

\begin{equation}\label{eq:maututility}
utility(A,u_j)=\Sigma_{f=true \in A} \Sigma_{d \in Dims} u(f,d)\times up(u_j,d)
\end{equation}

\begin{table}[ht]
\centering 
\caption{Utilities of example configurations $A_1..A_3$ for users $u_a$ and $u_b$.}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
\hline
configuration                 &  $A_1$            & $A_2$         & $A_3$ 
\tabularnewline
\hline \hline
$utility(A_i,u_a)$            &   1.76       &       2.32       &     \bf2.76        
\tabularnewline
\hline
$utility(A_i,u_b)$           &    3.44     &      3.58       &     \bf4.44     
\tabularnewline
\hline
\end{tabular} 
\label{tab:utilitycalculations}
\end{table}


Utility-based ranking (recommendation) can be used if \emph{alternative configurations have already been determined} \cite{Felfernig2006}. Note that we focused on a scenario where utility-based recommendation is used to identify a recommendation of relevance for a single user. However, there are also scenarios where recommendations have to be determined for \emph{groups of users} \cite{Felfernigetal2018}. In such a context, the preferences of individual users (group members) are aggregated, for example, by interpreting a group rating as the average value of the user-individual item evaluations. For details regarding the application of group recommender systems in configuration contexts we refer to \cite{Felfernigetal2016,Felfernigetal2018}.

Utility-based ranking has the disadvantage of \emph{knowledge acquisition efforts} that are needed to specify the contributions of user selections to individual interest dimensions. In the following section, we will discuss  scenarios in which recommender systems can be applied to support users in the selection of individual features, i.e., the configuration process is still ongoing and users need support in selecting and deselecting individual features.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Supporting Interactive Configuration}\label{configuration}

Assuming the availability of user interaction data from previous configuration sessions, we are able to proactively support the \emph{current user} interacting with a configurator \cite{Falkner2011,Pereira2018,RodasSilva2019}. A need for such a functionality is given if a user has \emph{limited domain knowledge} and is unsure about the inclusion or exclusion of a specific feature or a user simply \emph{does not have the time to specify every feature} (which is often the case with large and complex feature models). Table \ref{tab:usersessionsselectedfeatures} depicts a simplified example of such a scenario where users ($u_1..u_3$) have already completed their configuration sessions. The $current$ user just started his/her session and has specified his/her preferences regarding the features \emph{lic} (\emph{license}), \emph{adlic} (\emph{advanced license}), and \emph{baslic} (\emph{basic license}). 

\begin{table}[ht]
\centering 
\caption{Recommendation of a feature value ($0$) for feature $ABtesting$ to the current user.}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
\hline
  user/   &  lic  & adlic   & baslic     & AB    & stat          & QA      &bas-     &   mm-
\tabularnewline
 session   &    &    &      &     &           &       &  QA   &  QA 
\tabularnewline
\hline \hline
$u_1$           &   1   &  0    &  1        &  0    &     1         &  1        &   1   &    0
\tabularnewline
\hline
$u_2$           &   1   &  1    &  0        &  1    &     1         &  1        &   1   &    1
\tabularnewline
\hline
$u_3$           &   1   &  0    &  1        &  0    &     1         &  1        &   1   &    0
\tabularnewline
\hline
current     &   1   &  0    &  1        &  ?$\rightarrow$0    &     -         &  -        &   -   &    -
\tabularnewline
\hline
\end{tabular}
\label{tab:usersessionsselectedfeatures} 
\end{table}



The idea of \emph{collaborative filtering} based recommendation \cite{Konstan1997} is to analyze  users with similar preferences compared to the current user (\emph{nearest neighbors}) and exploit this knowledge for determining recommendations for the \emph{current user}.\footnote{For a detailed analysis of different collaborative filtering approaches, we refer to Ekstrand et al. \cite{Ekstrand2011}.} For example, the users with the most similar preferences compared to the current user are $u_1$ and $u_3$. If we are interested in recommending feature inclusion or exclusion for the feature $ABtesting$ ($AB$), we could recommend to the current user to follow his/her nearest neighbors $u_1$ and $u_3$, i.e., not to include this feature. In such scenarios, the number of nearest neighbors can be regarded as hyper-parameter which can be tuned during the learning phase of a recommendation algorithm.


A basic example similarity function which helps to figure out the similarity between user $u_a$ and $u_b$ is represented by Formula \ref{usersimilarity}. In this context, the set $F$ represents those features that have been specified, i.e., \emph{selected or deselected} by both users. 

\begin{equation}\label{usersimilarity}
    sim(u_a,u_b)=\frac{|\{f \in F: f(u_a)=f(u_b)\}|}{|\{f \in F\}|}
\end{equation}

In Table \ref{tab:usersessionsselectedfeatures}, the similarity between the current user and user $u_1$ is $1.0$, i.e., both users have completely the same preferences. In the mentioned scenario, we have to support \emph{session-based recommendation} \cite{Wang2019ASO}, since recommendations are determined by using preference information from the \emph{current user session} and the preferences of similar users. We want to emphasize that in real-world scenarios nowadays model-based machine learning approaches such as matrix factorization \cite{Koren2009} are applied for item prediction. An example of applying matrix factorization is provided in Section \ref{reconfiguration}. These approaches manage to encode complex similarity relationships into a set of \emph{hidden aspects}.

Up to now, we focused on recommending the \emph{selection or deselection of individual features}. A similar recommendation approach can also be applied to the \emph{selection of the next choice point}, i.e., which feature(s) should be presented next to the user for a selection/deselection decision. In this scenario, we recommend the next feature (the next features) a user could be interested in to specify. Note that in such scenarios information gain is often not the best criteria for attribute selection since users do not necessarily follow the criteria of information gain when selecting the next feature.


\begin{table}[ht]
\centering 
\caption{Recommendation of a possible next feature ($QA$) to be specified by the current user.}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
\hline
  user/   &  lic  & adlic   & baslic     & AB    & stat          & QA      &bas-     &   mm-
\tabularnewline
 session   &    &    &      &     &           &       &  QA   &  QA 
\tabularnewline
\hline \hline
$u_1   $        &   1   &  3    &  2        &  4    &     5         &  6        &   8   &    7
\tabularnewline
\hline
$u_2$           &   2   &  3    &  4        &  1    &     8         &  5        &   7   &    6
\tabularnewline
\hline
$u_3$           &   1   &  2    &  3        &  5    &     8         &  4        &   7   &    6
\tabularnewline
\hline
current     &   1   &  2    &  3        &  ?    &     ?         &  ?$\rightarrow$4        &   ?   &    ?
\tabularnewline
\hline
\end{tabular}
\label{tab:userssessionfeatureordering}
\end{table}


In the example shown in Table \ref{tab:userssessionfeatureordering}, the current user has already specified his/her preferences regarding \emph{license} ($lic$), \emph{advanced license} ($adlic$), and \emph{basic license} ($baslic$). Now, we are interested in which feature the user would like to specify next. Formula \ref{userrankingsimilarity} is a variation of Formula \ref{usersimilarity} where similarity measurement focuses on the distance of feature selection orderings of two different users $u_a$ and $u_b$.

\begin{equation}\label{userrankingsimilarity}
    sim(u_a,u_b)=\frac{m(u_a,u_b) - \Sigma_{f \in F} |fr(u_a,f) - fr(u_b,f)|}{m(u_a,u_b)}
\end{equation}

 In Formula \ref{userrankingsimilarity}, $F$ denotes features that have already been specified by both users. The function $fr(u,f)$ denotes the selection position (rank) of feature $f$ in the session of user $u$.  Furthermore, the function $m$ denotes the maximum possible distance between $u_a$ and $u_b$ in terms of the order of user feature specifications. For example, $m(current,u_3)$ is $(|1-3|)+(|2-2|)+(|3-1|)$ = $2+0+2=4$ assuming the initial feature selection ordering 1:$lic$, 2:$adlic$, and 3:$baslic$. Consequently, $sim(current, u_3)$ = $\frac{4-0}{4}$ = $1.0$.
 
 After having identified the most similar user(s) of the current user, the next feature to be specified can be recommended. This is the feature with lowest ranking of the similar users that has not been specified up to now by the current user. In our example, feature $QA$ would be the recommendation candidate since the most similar user of the current user is \emph{$u_3$} and the lowest ranking of a feature not specified by the \emph{current user} is $4$ which represents $QA$ in the case of  $u_3$.
 
 \vspace{0.25cm}
 
 \textbf{Research Issues}. We want to emphasize that the mentioned recommendation approaches are not able to guarantee that the determined recommendation is \emph{consistent with the constraints in the feature model}, i.e., it could be the case that a feature setting is recommended which is \emph{inconsistent} with the underlying feature model. Basically, this means that the recommender system is unaware of the constraints defined in the feature model. A possibility to avoid such a situation is to trigger an additional consistency check before the recommendation is shown to the user. However, this requires additional computing time and could in some cases result in slow response times. An alternative is to interpret recommendations as \emph{search heuristics} (e.g., variable value orderings) and to determine recommendations (configurations) on the solver level. An initial approach to  recommendation-based search  is discussed in detail in \cite{PolatErdeniz2019}. A major challenge is to learn configurator search heuristics in such a way that a reasonable \emph{tradeoff between prediction quality and search effort} can be achieved. For related work on search optimization in feature model related reasoning we refer, for example, to Sayyad et al. \cite{Sayyad2013}.
 

 
 



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Supporting Reconfiguration}\label{reconfiguration}

With \emph{reconfiguration} \cite{Janota2014} we refer to (often interactive) scenarios where (1) a set of features has already been specified (selected or deselected) by a user but triggers an inconsistency or (2) additional features have been specified in the feature model and we would like to know ahead which user/customer is interested in extending his/her current configuration, i.e., including the new feature.

An example of the first scenario is depicted in Table \ref{tab:reconfigurationinconsistency}. The \emph{current user} has changed his/her mind and thinks about choosing a basic license ($baslic$). At the same time, it seems to be the case that the user is still interested in having an \emph{ABtesting} ($AB$) support. In this context, we have to indicate to the user which of his/her  preferences have to be adapted to restore consistency.

\begin{table}[ht]
\centering 
\caption{Log of already completed configurations (conf). The current user has specified inconsistent preferences (if $ABtesting$ is selected, a license has to be paid).}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
\hline
  user/   &  lic  & adlic   & baslic     & AB    & stat          & QA      &bas-     &   mm-
\tabularnewline
conf   &    &    &      &     &           &       & QA    & QA  
\tabularnewline
\hline \hline
$u_1$           &   1   &  0        &  1        &  0            &     1         &  1        &   1   &    0
\tabularnewline
\hline
$u_2$           &   1   &  1        &  0        &  1            &     1         &  1        &   1   &    1
\tabularnewline
\hline
$u_3$           &   1   &  0         &  1        &  0           &     1         &  1        &   1   &    0
\tabularnewline
\hline
current     & 1     &  \bf0!     &  \bf1!     &  \bf1!            &     1         &     1     &   1   &    1
%current     & 1     &  \bf0!     &  \bf1!     &  1?            &     1?         &     1?     &   1?   &    1?
\tabularnewline
\hline
\end{tabular}
\label{tab:reconfigurationinconsistency} 
\end{table}

Technically speaking, there exist two conflicts ($CS_{1,2}$) \cite{Junker2004} between the mentioned user preferences: $CS_1: \{adlic=0,AB=1\}$ and $CS_2: \{baslic=1,AB=1\}$. This inconsistency could be resolved by pointing out to the user the option of restoring the original setting that accepts advanced licensing (alt. $1$: $adlic=1$, $baslic=0$, $AB=1$) or to accept the reduction in functionality in terms of not having available anymore  \emph{ABtesting} (alt. $2$: $adlic=0$, $baslic=1$, $AB=0$).






If more information is available about the preferences of the user, for example, we know that a user is  more interested in a simple solution ($simplicity=0.8$) compared to a full feature support ($productivity = 0.2$), we are able to rank the individual alternatives of restoring consistency. By reusing the utility scores specified in Table \ref{tab:interestdimensionsattributes}, we are able to determine a ranking for the individual change recommendations possible in the mentioned scenario (see Table \ref{tab:rankingrepairs}).

\begin{table}[ht]
\centering 
\caption{Utility evaluation of change alternatives (based on Table \ref{tab:interestdimensionsattributes}). For example, in the context of $alternative~1$ and $licensing$, $1(0.1,1)$ denotes the fact that the inclusion of licensing contributes $0.1$ to simplicity and $1.0$ to productivity.}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
\hline
  preference    &  advanced-                  & basiclicense       & ABtesting    & utility
\tabularnewline
 change   &  license                  &        &     & 
\tabularnewline
\hline \hline
alt. 1                 &   $1(0.1,1)$     &  $0(1,0.1)$  &   $1(0.3,1)$  & 0.72
\tabularnewline
\hline
alt. 2                 &   $0(0.1,1)$       &  $1(1,0.1)$  & $0(0.3,1)$ &   0.82
\tabularnewline
\hline
\end{tabular}
\label{tab:rankingrepairs} 
\end{table} 


Taking into account the preferences of our example user ($u_a$), the utility of change $alt.~1$ is $(0.1*0.8+1.0*0.2)+(0.3*0.8+1.0*0.2)=0.72$. Furthermore, the overall utility of change $alt.~2$ is $1.0*0.8+0.1*0.2=0.82$. Consequently, we could recommend change $alt.~2$ to the user which clearly focuses on the aspect of $simplicity$. Determining conflicts and corresponding change alternatives is the task of \emph{conflict detection} and corresponding \emph{diagnosis} algorithms. A discussion of these algorithms is beyond the scope of this paper. For further related details regarding conflict detection and diagnosis we refer to Junker \cite{Junker2004}, Reiter \cite{Reiter1987}, and Felfernig et al. \cite{Felfernig2012}.


Reconfiguration is not only associated with consistency management but can also be relevant when existing configurations should be extended with additional features. In Table \ref{tab:featureextension}, an additional feature \emph{share} has been added which supports the sharing of the results of a \emph{survey}. An important information in this context (e.g., in marketing scenarios) is to know which users would be interested in the new feature and should be primarily contacted. 

\begin{table}[ht]
\centering 
\caption{Session log of already completed configurations.}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|}
\hline
  user   &  lic  & ad-   & bas-     & AB    & stat          & QA      &bas-     &   mm- & share
\tabularnewline
   &    & lic   & lic     &     &           &       &QA     &   QA & 
\tabularnewline
\hline \hline
$u_1$           &   1   &  0    &  1    &  0    &     1         &  1        &   1   &    0      & 0
\tabularnewline
\hline  
$u_2$           &   1   &  1    &  0    &  1    &     1         &  1        &   1   &    1      & 1
\tabularnewline
\hline
$u_3$           &   1   &  0    &  1    &  0    &     1         &  1        &   1   &    0      & ?
\tabularnewline
\hline
$u_4$     &   1   &  0    &  1    &  1    &     1         &  1        &   1   &    0      &     ?
\tabularnewline
\hline
\end{tabular}
\label{tab:featureextension} 
\end{table}



The \emph{relevance prediction} for a new feature for a user can be implemented with collaborative filtering (CF) \cite{Konstan1997}. A major difference compared to the previously discussed CF approaches is that the relevance of new features can be easily determined \emph{offline} which makes the task more appropriate for model-based collaborative filtering often implemented as \emph{matrix factorization} (MF) \cite{Koren2009}. MF focuses on optimizing a set of so-called hidden aspects which can then be used to predict the preferences of individual users.

Table \ref{tab:featureextension} ($T$) can be reconstructed using \emph{dimensionality reduction} which is based on the idea of learning two low-dimensional matrices ($UA$ and $AF$) that help to derive a matrix $T'$ \textasciitilde $~T$, i.e., $T'$ approximates $~T$. The advantage of this approach is generalizability and efficiency since we are able to predict individual  preferences on the basis of a simple matrix multiplication operation. Let us first construct the matrices \emph{UA} and \emph{AF} on the basis of the evaluation dimensions \emph{productivity} and \emph{simplicity} as depicted in Tables \ref{tab:matrixproductivity} and \ref{tab:matrixsimplicity}.

When applying matrix factorization to the matrices \emph{UA} and \emph{AF}, we can derive the matrix \emph{T'} which is an approximation of the original matrix \emph{T}. Note that up to now we just \emph{manually estimated} the relationship between user- and item-specific interest dimensions. The disadvantage of this approach is that estimation has to be performed manually with tedious adaptation efforts for new features and also the requirement that each user has to specify his/her preferences regarding the interest dimensions. For the new feature \emph{share} (see Table \ref{tab:matrixsimplicity}), we assume a high evaluation with regard to the dimension \emph{productivity} (additional functionality is provided) and a very high evaluation with regard to  \emph{simplicity} (the \emph{share} feature does not trigger additional complexity).

Using matrix factorization, this manual process can be substituted by machine learning that estimates the weights of individual interest dimensions to optimize the similarity between $T'$ and $T$ \cite{Koren2009}. Table \ref{tab:matrixresulting} represents the result of   a matrix multiplication \text{UA} $\bullet$ \text{AF}. In this context, \emph{share} has very high estimate ($0.94$ on a scale $0..1$) for user $u_b$. Consequently, the new feature has a high chance to be of relevance for $u_b$.



\begin{table}[ht]
\centering 
\caption{Matrix UA representing user preferences regarding interest dimensions (aspects).}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|}
\hline
  user   &  $productivity$ & $simplicity$
\tabularnewline
\hline \hline
  $u_a$   &  0.8 & 0.2
\tabularnewline
\hline
$u_b$   &  0.2 & 0.8
\tabularnewline
\hline
\end{tabular}
\label{tab:matrixproductivity}
\end{table}




\begin{table*}[ht]
\centering 
\caption{Matrix AF representing item property (feature) relationships to interest dimensions (aspects).}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|}
\hline
  dimension         & $adlic$   & $baslic$     & $AB$        & $stat$      &$mmQA$     &   $basQA$  & $share$
\tabularnewline
\hline \hline
$productivity$       & 1.0    &  0.1        & 1.0        & 1.0        & 1.0    & 1.0    &0.7
\tabularnewline
\hline
$simplicity$           & 0.1    &  1.0        & 0.3        & 0.5        & 0.3    & 1.0    &1.0
\tabularnewline
\hline
\end{tabular}
\label{tab:matrixsimplicity}
\end{table*}




\begin{table}[ht]
\centering 
\caption{Matrix $T'$ resulting from a matrix multiplication \text{UA} $\bullet$ \text{AF}. Feature \textsc{share} appears to be potentially relevant for $u_b$.}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|}
\hline
  user         & $adlic$   & $baslic$     & $AB$        & $stat$      &$mmQA$     &   $basQA$  & $share$
\tabularnewline
\hline \hline
$u_a$       & 0.82    &  0.28        & 0.82        & 1        & 1    & 1         &0.76
\tabularnewline
\hline
$u_b$           & 0.28    &  0.82        & 0.28        & 1        & 1    & 1    &\bf0.94 
\tabularnewline
\hline
\end{tabular}
\label{tab:matrixresulting} 
\end{table}


\begin{table}[ht]
\centering 
\caption{Matrix $T'$ resulting from a matrix multiplication \text{UA} $\bullet$ \text{AF}. Feature \textsc{share} appears to be potentially relevant for $u_b$.}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|}
\hline
  user         & $adlic$   & $baslic$     & $AB$        & $stat$      &$mmQA$     &   $basQA$  & $share$
\tabularnewline
\hline \hline
$u_a$       & 1    &  0        & 1        & 1        & 1    & 1         &0
\tabularnewline
\hline
$u_b$           & 0    &  1        & 0        & 1        & 1    & 1    &\bf1 
\tabularnewline
\hline
\end{tabular}
\label{tab:matrixcorrect} 
\end{table}


Such a machine learning approach substitutes manual preference specification but also has the \emph{disadvantage} that the learned dimensions (aspects) do not have a clear semantics but are just representing abstract properties that optimize the prediction quality of user interests regarding individual features. When applying matrix factorization, the interest dimensions of Tables \ref{tab:matrixproductivity} and \ref{tab:matrixsimplicity} would be substituted by two (or more) optimized abstract dimensions \cite{Koren2009}.

Many software systems are configurable in one way or another \cite{Pereira2020spaces,Pereira2020,Temple2017}. In many cases, the configuration space is huge and  mechanisms are needed that help to support tasks such as the prediction of the performance of specific (re-)configurations and the optimization of configurations. \emph{Performance prediction} can play an important role in the context of (re-)configuring packages and parameters of an operating system. In this context, it should be possible to predict system performance to avoid slow runtimes due to low-quality parametrizations.

A system should also be able to support parameter optimization, i.e., to recommend reasonable parameter settings during the ramp-up phase of a system or during reconfiguration. Such an optimization can be performed following so-called \emph{sampling, measuring, and learning} patterns \cite{Pereira2020spaces} with the overall task of identifying representative system parameter settings, measuring the impact of a specific configuration, and learn a general model to be able to classify between high- and low-quality parametrizations (configurations). 

The mentioned goals can be supported a.o. on the basis of matrix factorization \cite{Koren2009} where samples can be regarded as reference points and factorization can be applied to (1) recommend parameter settings that will result in a good system performance and (2) provide hints that some of the  parametrizations will lead to low system performance.


\vspace{0.25cm}


\textbf{Research Issues}. Similar to collaborative filtering, recommendations determined by matrix factorization cannot guarantee the feasibility of a recommendation, i.e., there can be situations where a recommendation induces an inconsistency with the constraints defined in the feature model. As mentioned, an approach to deal with such situations is to recommend solver search heuristics that are learned from existing user interaction data \cite{PolatErdeniz2019}. An important issue related to both scenarios, i.e., configuration and reconfiguration, is how to \emph{explain recommendations}. In both scenarios, explanations are shallow, i.e., only refer to the used algorithm. For example, collaborative filtering is based on explanations of the preferences of nearest neighbors. A direction for future research in this context is to combine machine learning with less-well performing knowledge-based recommendation approaches (e.g., the utility-based approach discussed in Section \ref{configurationranking}) and then generate explanations on the basis of the knowledge-based recommendation model. Such knowledge-based models allow for a more fine-grained explanation on the semantic level. An example research issue is to figure out which criteria are sufficient for the application of an inferior (in terms of prediction quality) knowledge-based approach for the generation of explanations.



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Supporting Modeling Processes}\label{modeling}

Finally, we take a look at the process of feature model development. In this context, recommendation approaches can be applied, for example, in the context of \emph{learning processes}. Engineers of feature models who are in charge of overtaking the development and maintenance of a new feature model, often need support in the navigation of the feature and constraint space. A basic idea to support such learning processes is to apply collaborative filtering which can help to recommend items (e.g., constraints) that could be of relevance for the knowledge engineer at a specific point of time. The recommendation approach that can be applied in this context is quite similar to the one discussed in the context of recommending features to be specified within the scope of configuration sessions. In the example depicted in Table \ref{tab:sessionspecificconstraintedits}, the current knowledge engineer ($ke$) interacting with the feature model has already visited/edited the constraints $c_1..c_3$. Collaborative recommendation can be applied to predict further relevant constraints he/she could take a look at. In our example, this would be the constraint $c_4$.

\begin{table}[ht]
\centering 
\caption{Session-specific ordering of constraint edits.}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|}
\hline 
  session/ke   &  $c_1$& $c_2$ &  $c_3$    & $c_4$    & $c_5$    & $c_6$      &$c_7$     &   $c_8$ & $c_9$
\tabularnewline
\hline \hline
$1$           &   1   &  3    &  2        &  4    &     5         &  6        &   8   &    7 & 9
\tabularnewline
\hline
$2$           &   2   &  3    &  4        &  1    &     8         &  5        &   7   &    9 & 6
\tabularnewline
\hline
$3$           &   1   &  2    &  3        &  4    &     9         &  5        &   7   &    6 & 8
\tabularnewline
\hline
current     &   1   &  2    &  3        &  ?$\rightarrow$4    &     ?         &  ?        &   ?   &    ? & ?
\tabularnewline
\hline
\end{tabular} 
\label{tab:sessionspecificconstraintedits} 
\end{table}

Recommending the next constraint is relevant to support knowledge engineers in understanding a knowledge base. A related aspect is the \emph{grouping of constraints} in such a way that the cognitive overload of knowledge engineers can be minimized. One way to achieve this is to apply the concepts of clustering which helps to identify constraints which belong together in one way or another. An analysis of different approaches to group constraints in configuration knowledge acquisition contexts can be found, for example, in Felfernig et al. \cite{Felfernigetal2013}.


Besides supporting users in understanding a knowledge base (feature model), recommender systems can also be applied to automatically generate a knowledge base. Ulz et al. \cite{Ulz2017} introduce a \emph{human computation}\footnote{The underlying idea of \emph{human computation} is that humans take over problem solving tasks which computers are not able to solve in equal quality.} \cite{vonAhn2005} based approach where the development of recommender and configuration knowledge bases is implemented on the basis of asking users simple questions and aggregate the results in an intelligent fashion into a corresponding set of constraints. Questions refer to selection scenarios, for example, users should give feedback, which items they would like to be included in a recommendation in a specific context. The output of the approach is a set of \emph{requires} constraints that support item selection in different contexts. Bécan et al. \cite{Becan2015} and She et al. \cite{She2014} follow the similar idea of generating feature models from configuration instances, i.e., instead of asking the user questions about intended item properties, the intended properties are already represented as configurations or logical formulae.

\vspace{0.25cm}

\textbf{Research Issues}. Feature model development should be accompanied by structured testing approaches which support quality assurance of feature models, i.e., to assure that the structures and constraints defined in the feature model are consistent with the underlying domain knowledge. For example, \emph{it should not be possible} to calculate configurations that (1) include features that should not be combined with each other and (2) exclude features that are regarded as feasible in the underlying application domain. 

Quality assurance for feature models is already supported by different types of analysis operations (see, e.g., \cite{BeSeRu2010}). Open issues in this context are the \emph{recommendation of relevant test cases} useful for identifying erroneous constraints in a feature model and the recommendation of corresponding diagnoses that indicate minimal subsets of constraints which could be responsible for the faulty behavior of the feature model. Such a recommendation support would help to further improve the efficiency of feature model knowledge acquisition and maintenance processes.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Conclusions and Further Research Issues}\label{conclusions}

With this paper, we provide an overview of feature modeling and configuration scenarios that can profit from the application of recommendation and related machine learning approaches. For selected scenarios, we have provided examples that help to improve the understanding of the discussed application.  Future work in this context includes a couple of empirical evaluations that will help to better estimate which recommendation approach best supports a specific scenario. There are a couple of further scenarios that could profit from the application of recommendation technologies. 

For example, the analysis of user interaction data could indicate \emph{additional relevant constraints to be included in a model}. These constraints can also be regarded as specializations of an existing knowledge base \cite{Temple2016}. This functionality could be based on association rule mining applied to the interaction data. Such an approach follows the line of research of learning whole feature models from pre-existing configurations (see, for example, Bécan et al. \cite{Becan2015}).

In knowledge acquisition scenarios, an \emph{intelligent grouping of features and constraints of a feature model} could be important to streamline maintenance processes. Supporting such a grouping can, for example, be based on clustering approaches as discussed in Felfernig et al. \cite{Felfernigetal2013}. A major issue for future work in this context is to make constraint groups flexible and adaptive to different scenarios, for example, searching for a faulty constraint or changing a specific variability property in a feature model.

A related challenge is how to recommend maintenance actions to \emph{avoid low-quality modeling} in terms of complex hierarchical structures and constraints of low understandability. Such recommendations can also be based on a more in-depth knowledge of cognitive aspects in knowledge representation and maintenance.  

Finally, an open issue in the context of applying recommendation technologies in complex item domains is how to evaluate the quality of  recommendations. Existing evaluation measures have to be adapted or extended. An  example is the measurement of \emph{precision}: in the configuration context, single features or groups of feature settings could be recommended.

\begin{acks}

The work presented in this paper has been developed within the research project \textsc{ParXCel} (\emph{Machine Learning and Parallelization for Scalable Constraint Solving}) which is funded by the Austrian Research Promotion Agency (FFG) under the project number $880657$. We want to thank the following persons for their valuable comments which helped to further improve the quality of this paper: David Benavides (University of Seville), Mayte Gómez (University of Seville), and Klaus Pilsl (Combeenation).

\end{acks}



\bibliographystyle{ACM-Reference-Format}
\bibliography{bibliography}


\end{document}

bibliography.bib
@ARTICLE{felfernig00,
  author = {Felfernig, A. and Friedrich, G.~E. and Jannach, D.},
  title = {{UML as Domain Specific Language for the Construction of Knowledge-based
	Configuration Systems}},
  journal = {International Journal of Software Engineering and Knowledge Engineering},
  year = {2000},
  volume = {10},
  pages = {449-469},
  number = {4},
  publisher = {World Scientific Publishing Company}
}

@ARTICLE{Felfernig2007,
  author = {A. Felfernig},
  title = {Standardized Configuration Knowledge Representations as Technological
	Foundation for Mass Customization},
  journal = {IEEE Transactions on Engineering Management},
  year = {2007},
  volume = {54},
  pages = {41-56}
}

@ARTICLE{She2014,
  author = {S. She and U. Ryssel and N. Andersen and A. Wasowski and C. Czarnecki},
  title = {Efficient synthesis of feature models},
  journal = {Information and Software Technology},
  year = {2014},
  volume = {56},
  number = {9},
  pages = {1122--1143}
}



@inproceedings{betty,
 author = {S. Segura and J. Galindo and D. Benavides and J. Parejo and A. Ruiz-Cort{\'e}s},
 title = {{BeTTy: Benchmarking and Testing on the Automated Analysis of Feature Models}},
 booktitle = {{6th International Workshop on Variability Modeling of Software-Intensive Systems}},
 series = {6th International Workshop on Variability Modelling of Software-intensive Systems (VaMoS'12)},
 year = {2012},
 location = {Leipzig, Germany},
 pages = {63--71},
 publisher = {ACM},
 address = {New York, NY, USA}
} 


@inproceedings{Sayyad2013,
 author = {A. Sayyad and J. Ingram and T. Menzies and H. Ammar},
 title = {{Scalable product line configuration: A straw to break the camel's back}},
 booktitle = {{28th IEEE/ACM International Conference on Automated Software Engineering (ASE)}},
 year = {2013},
 pages = {465--474},
 address = {Silicon Valley, CA, USA}
} 


@inproceedings{Becan2015,
 author = {G. Bécan and R. Behjati and A. Gotlieb and M. Acher},
 title = {{Synthesis of attributed feature models from product descriptions}},
 booktitle = {{19th International Software Product Line Conference}},
 year = {2015},
 pages = {1--10},
 address = {Nashville, TN, USA}
} 




 



@inproceedings{Pereira2020,
 author = {J. Pereira and H. Martin and P. Temple and M. Acher},
 title = {{Machine Learning and Configurable Systems: A Gentle Introduction}},
 booktitle = {{24th ACM Systems and Software Product Line Conference}},
 series = {SPLC},
 year = {2020},
 pages = {Vol. A},
 number = {40},
 publisher = {ACM}
} 

@inproceedings{Pereira2020spaces,
 author = {J. Pereira and H. Martin and M.  Acher and J-M. Jézéquel and G. Botterweck and A. Ventresque},
 title     = {Learning Software Configuration Spaces: {A} Systematic Literature
               Review},
  journal   = {CoRR},
  volume    = {abs/1906.03018},
  year      = {2019},
  archivePrefix = {arXiv}
} 



@ARTICLE{ZellerIEEEComputer2001,
  author = {A. Zeller},
  title = {Automated debugging: are we close?},
  journal = {IEEE Computer},
  year = {2001},
  volume = {34},
  number = {11},
  pages = {26--31}
}

@ARTICLE{Fraser2009,
  author = {G. Fraser and A. Zeller},
  title = {Mutation-Driven Generation of Unit Tests and Oracles},
  journal = { IEEE Transactions on Software Engineering},
  year = {2012},
  volume = {38},
  number = {2},
  pages = {278--292}
}


@ARTICLE{Zeller2002,
  author = {A. Zeller and R. Hildebrandt},
  title = {Simplifying and Isolating Failure-Inducing Input},
  journal = {IEEE Transactions on Software Engineering},
  year = {2002},
  volume = {28},
  number = {2},
  pages = {183--200}
}


@article{Trinidad2008,
	author = {P. Trinidad and D. Benavides and A. Duran and A. Ruiz-Cortez and M. Toro},
	title = {{Automated error analysis for the agilization of feature modeling}},
	journal = {Journal of Systems and Software},
	volume = {81},
	year = {2008},
	pages = {883--896},
	publisher = {Elsevier}
} 


@article{BeSeRu2010,
	author = {D. Benavides and S. Segura and A. Ruiz-Cortes},
	title = {{Automated analysis of feature models 20 years later: A literature review}},
	journal = {Information Systems},
	volume = {35},
	issue = {6},
	year = {2010},
	pages = {615--636},
	publisher = {Elsevier}
}


@article{FFJ2000,
	author = {A. Felfernig and G. Friedrich and D. Jannach},
	title = {{UML ad domain specific language for the construction of knowledge-based configuration systems}},
	journal = {International Journal on Software Engineering and Knowledge Engineering},
	volume = {10},
	number = {4},
	year = {2000},
	pages = {449--470},
	publisher = {World Scientific Publishers}
}



@article{LiKo2008,
	title = {{Multi-objective optimization of generalized reliability design problems using feature models -- A concept for early design stages}},
	journal = "Reliability Engineering \& System Safety",
	volume = "93",
	number = "6",
	pages = "815 - 828",
	year = "2008",
	note = "",
	issn = "0951-8320",
	doi = "10.1016/j.ress.2007.03.032",
	url = "http://www.sciencedirect.com/science/article/pii/S0951832007001299",
	author = "P. Limbourg andvH. Kochs"
}


@article{Kang1990,
	title = {{Feature-oriented Domain Analysis (FODA) -- Feasibility Study}},
	journal = "TechnicalReport CMU -- SEI-90-TR-21",
	publisher = "SoftwareEngineeringInstitute",
          address = "Carnegie Mellon University, Pittsburgh, PA, USA",
	year = "1990",
	author = {K. Kang and S. Cohen and J. Hess and W. Novak and S. Peterson}
}






@article{Pazzani1997,
	title = {{Learning and revising user profiles: The identification of
interesting web sites}},
	journal = "Machine Learning",
	volume = {27},
	year = "1997",
	pages = {313--331},
	author = {M. Pazzani and D. Billsus}
}



@article{Burke2000,
	title = {{Knowledge-based recommender systems}},
	journal = "Encyclopedia of Library and
Information Systems",
	volume = {69},
	number = {32},
	year = "2000",
	pages = {180--200},
	author = {R. Burke}
}




@article{Gomes2008,
           authors = {C. Gomes and H. Kautz and A. Sabharwal and B. Selman},
	title = {{Satisfiability Solvers}},
	journal = "Handbook of Knowledge Representation",
           series = "Foundations of Artificial Intelligence",
	publisher = "Elsevier",
          editors = "F. VanHarmelen and Vladimir Lifschitz and Bruce Porter",
          number = "3",
	year = "2008",
          pages = "89--134"
}


@article{Konstan1997,
    author = {J. Konstan and B. Miller and J. Herlocker and L. Gordon and J. Riedl},
	title = {{GroupLens: Applying Collaborative Filtering to Usenet News}},
	journal = "Communications of the ACM",
	publisher = "ACM",
	volume = {40},
    number = "3",
	year = "1997",
    pages = "77--87"
}



@article{Pereira2018,
    author = {J. Pereira and P. Matuszyk and S. Krieter and M. Spiliopoulou and G. Saake},
	title = {{Personalized Recommender Systems for Product-Line Configuration Processes}},
	journal = "Computer Languages, Systems \& Structures",
	publisher = "Elsevier",
	volume = {54},
	year = "2018",
    pages = "451--471"
}






@article{Wang2019ASO,
  title={A Survey on Session-based Recommender Systems},
  author={S. Wang and L. Cao and Y. Wang},
  journal={arXiv},
  year={2019},
  volume={abs/1902.04864}
}


@article{Felfernig2006,
    author = {A. Felfernig and G. Friedrich and D. Jannach and M. Zanker},
	title = {{An Integrated Environment for the Development of Knowledge-based Recommender Applications}},
	journal = "Intl. Journal of Electronic Commerce",
	volume = {11},
    number = "2",
	year = "2006",
    pages = "11--34"
}



@article{Czarnecki2005,
	title = {{Formalizing Cardinality-based Feature Models and their Specialization}},
	journal = "SoftwareProcess: Improvement and Practice",
	year = "2005",
	author = "K. Czarnecki and S. Helsen and U. Eisenecker",
          volume = "10",
          number = "1",
          pages = "7--29"
}


@incollection {Ba2005,
	author = {D. Batory},
	title = {{Feature Models, Grammars, and Propositional Formulas}},
	booktitle = {Software Product Lines Conference},
	series = {LNCS},
	editor = {H. Obbink and K. Pohl},
	publisher = {Springer},
	pages = {7-20},
	volume = {3714},
	year = {2005}
}


@article{Trinidad2008,
	author = {Y. Koren and R. Bell and C. Volinsky},
	title = {{Automated error analysis for the agilization of feature modeling}},
	journal = {The Journal of Systems and Software},
	volume = {81},
	issue = {6},
	year = {2008},
	pages = {883--896},
	publisher = {Elsevier},
	address = {New York, NY, USA}
}


@article{Koren2009,
	author = {Y. Koren and R. Bell and C. Volinsky},
	title = {{Matrix factorization techniques for recommender systems}},
	journal = {IEEE Computer},
	volume = {42},
	issue = {8},
	year = {2009},
	pages = {30--37}
}



@article{Reiter1987,
	author = {R. Reiter},
	journal = {Artificial Intelligence},
	pages = {57--95},
	publisher = {Elsevier Science Publishers Ltd.},
	title = {A theory of diagnosis from first principles},
	volume = 32,
           number = 1,
	year = 1987
}


@article{Apel2009,
	author = {S. Apel and C. Kästner},
	journal = {Journal of Object Technology},
	pages = {49--84},
	title = {An Overview of Feature-Oriented Software Development},
	volume = 8,
           number = 5,
	year = 2009
}


@article{White2010,
	title = {{Automated diagnosis of feature model configurations}},
	journal = "Journal of Systems and Software",
	volume = "83",
	number = "7",
	pages = "1094--1107",
	year = "2010",
	author = "J. White and D. Benavides and D. Schmidt and P. Trinidad and B. Dougherty and A. Ruiz-Cortes"
}


@article{Temple2017,
	title = {{Learning Contextual-Variability Models}},
	journal = "IEEE Software",
	volume = "34",
	number = "6",
	pages = "64--70",
	year = "2017",
	author = "P. Temple and M. Acher and J-M. Jézéquel and O. Barais"
}


@inproceedings{Janota2014,
author = {M. Janota and G. Botterweck and J. Marques-Silva},
year = {2014},
pages = {8:1-8:8},
publisher = {ACM},
booktitle = {8th International Workshop on Variability Modelling of Software-intensive Systems (VaMoS'14)},
address = {Sophia Antipolis, France},
title = {On lazy and eager interactive reconfiguration},
}


@inproceedings{Temple2016,
author = {P. Temple and J. Galindo and M. Acher and J-M. Jézéquel},
year = {2016},
pages = {209--218},
publisher = {ACM},
booktitle = {20th International Systems and Software Product Line Conference},
address = {Beijing, China},
title = {Using machine learning to infer constraints for product lines},
}


@inproceedings{PolatErdeniz2019,
author = {S. Polat-Erdeniz and A. Felfernig and R. Samer and M. Atas},
year = {2019},
pages = {1655--1662},
publisher = {ACM},
booktitle = {ACM Symposium on Applied Computing (ACM SAC)},
address = {Limassol, Cyprus},
title = {{Matrix Factorization based Heuristics for Constraint-based Recommenders}},
}




@book{Tsang1993,
	title = " Foundations of Constraint Satisfaction",
	publisher = " Academic Press",
          address ="London",
	year = "1993",
	author = "E. Tsang"
}


@book{Felfernigetal2018,
	title = "Group Recommender Systems",
	publisher = " Springer",
	year = "2018",
	author = {A. Felfernig and L. Boratto and M. Stettinger and M. Tkalcic}
}

@book{vonAhn2005,
	title = "Human Computation",
	publisher = "Technical Report CM-CS-05-193",
	year = "2005",
	author = {L. von Ahn}
}


@book{Felfernigetal2014,
	title = "Knowledge-based Configuration - From Research to Business Cases",
	publisher = "Elsevier",
	year = "2014",
	author = "A. Felfernig and L. Hotz and C. Bagley and J. Tiihonen"
}


@book{Ekstrand2011,
	title = {{Collaborative Filtering Recommender Systems}},
	series = {{Foundations and Trends in Human–Computer Interaction}},
	volume = {4},
	number = {2},
	year = {2011},
	author = {M. Ekstrand and J. Riedl and J. Konstan}
}



@article{sabin98-is,
  author    = {Daniel Sabin and Rainer Weigel},
  title     = {Product Configuration Frameworks - A Survey},
  journal   = {IEEE Intelligent Systems},
  volume    = {13},
  number    = {4},
  year      = {1998},
  pages     = {42-49},
}


@article{fleischanderl1998,
  author    = {G. Fleischanderl and G. Friedrich and A. Haselboeck and H. Schreiner and M. Stumptner},
  title     = {Configuring Large Systems Using Generative Constraint Satisfaction},
  journal   = { IEEE Intelligent Systems},
  volume    = {13},
  number    = {4},
  year      = {1998},
  pages     = {59-68}
}


@article{Falkner2011,
  author    = {A. Falkner and A. Felfernig and A. Haag},
  title     = {{Recommendation Technologies for Configurable Products}},
  journal   = {AI Magazine},
  volume    = {32},
  number    = {3},
  year      = {2011},
  publisher = {AAAI},
  pages     = {99--108}
}



@article{RodasSilva2019,
  author    = {J. Rodas-Silva and J. Galindo and J. Garcia-Gutierrez and D. Benavides},
  title     = {{Selection of Software Product Line Implementation Components Using Recommender Systems: An Application to Wordpress}},
  journal   = {IEEE Access},
  volume    = {7},
  year      = {2019},
  publisher = {IEEE},
  pages     = {69226--69245}
}


@article{Ulz2017,
  author    = {T. Ulz and M. Schwarz and A. Felfernig and S. Haas and A. Shehadeh and S. Reiterer and M. Stettinger},
  title     = {{Human Computation for Constraint-based Recommenders}},
  journal   = {{Journal of Intelligent Information Systems}},
  volume    = {49},
  year      = {2017},
  publisher = {Springer},
  pages     = {37--57}
}






@inproceedings{Massen2004,
	author = {T. von der Massen and H. Lichter},
	booktitle = {Workshop on Software Variability Management for Product Derivation},
	editor = {T. Mannisto and J. Bosch},
	title = {{Deficiencies in Feature Models}},
	year = {2004}
}


@inproceedings{Benavidesetal2013,
	author = {D. Benavides and A. Felfernig and J. Galindo and F. Reinfrank},
	booktitle = {13th International Conference on Software Reuse (ICSR 2013)},
	title = {{Automated Analysis in Feature Modelling and Product Configuration}},
	address = {Pisa, Italy},
        pages = {160--175},
        series = {LNCS},
        number = {7925},
        year = {2013}
}



@inproceedings{FelfernigBurke2008,
	author = {A. Felfernig and R. Burke},
	booktitle = {ACM International Conference on Electronic Commerce},
	title = {{Constraint-based Recommender Systems: Technologies and Research Issues}},
	address = {Innsbruck, Austria},
        pages = {17--26},
        year = {2008}
}


@inproceedings{Benavidesetal2005,
	author = {D. Benavides and P. Trinidad and A. Ruiz-Cortez},
	booktitle = {International Conference on Advanced Information Systems Engineering},
	title = {{Automated Reasoning on Feature Models}},
	address = {Porto, Portugal},
	series = {{LNCS}},
	volume = {3520},
        pages = {491--503},
        year = {2005}
}

 

@article{Felfernig2012,
	title = "An Efficient Diagnosis Algorithm for Inconsistent Constraint Sets",
	journal = "AI for Engineering Design, Analysis, and  Manufacturing (AIEDAM)",
	volume = "26",
	number = "1",
	pages = "53--62",
	year = "2012",
           publisher = "Cambridge University Press",
	author = "A. Felfernig and M. Schubert and C. Zehentner"
}




@INPROCEEDINGS{Fleischanderl2002,
    author = {G. Fleischanderl},
    title = {Suggestions from the Software Engineering Practice for Applying Consistency-Based Diagnosis to Configuration Knowledge Bases},
    booktitle = {13th Intl. Workshop on Principles of Diagnosis (DX-02)},
    pages = {33--35},
    address = {Semmering, Austria},
    year = {2002}
}




@INPROCEEDINGS{Bakker93,
    author = {R.  Bakker and F. Dikker and F. Tempelman and P.  Wogmim},
    title = {Diagnosing and solving over-determined constraint satisfaction problems},
    booktitle = {Proceedings of IJCAI-93},
    year = {1993},
    pages = {276--281},
    publisher = {Morgan Kaufmann}
}





@INPROCEEDINGS{Felfernigetal2016,
    author = {A. Felfernig and M. Atas and T. Tran and M. Stettinger},
    title = {Towards Group-based Configuration},
    booktitle = {Workshop on Configuration (ConfWS'16)},
    year = {2016},
    pages = {69--72},
    address = {Toulouse, France}
}



@INPROCEEDINGS{Felfernigetal2013,
    author = {A. Felfernig and S. Reiterer and M. Stettinger and F. Reinfrank and M. Jeran and G. Ninaus},
    title = {Recommender Systems for Configuration Knowledge Engineering},
    booktitle = {Workshop on Configuration (ConfWS'13)},
    year = {2013},
    pages = {51--54},
    address = {Vienna, Austria}
}




@INPROCEEDINGS{FelfernigDX2011,
    author = {A. Felfernig and C. Zehentner and P. Blazek},
    title = {CoreDiag: Eliminating Redundancy in Constraint Sets},
    booktitle = {22nd International Workshop on Principles of Diagnosis},
    year = {2011},
    pages = {219--224},
    address = { Murnau, Germany}
}


% ---


@INPROCEEDINGS{AcCoLaFr2011,
	author={Acher, M. and Collet, P. and Lahire, P. and France, R.B.},
	booktitle={Automated Software Engineering (ASE), 2011 26th IEEE/ACM International Conference on},
	title={Slicing feature models},
	year={2011},
	month={nov.},
	volume={},
	number={},
	pages={424 -427},
	keywords={FM;SPL;feature models;novel slicing technique;software product lines;program slicing;software reusability;},
	doi={10.1109/ASE.2011.6100089},
	ISSN={},
}

@incollection {AcCoLaFr2010,
   author = {Acher, Mathieu and Collet, Philippe and Lahire, Philippe and France, Robert},
   affiliation = {University of Nice Sophia Antipolis I3S Laboratory (CNRS UMR 6070) France},
   title = {Comparing Approaches to Implement Feature Model Composition},
   booktitle = {Modelling Foundations and Applications},
   series = {LNCS},
   editor = {Kuehne, T. and Selic, B. and Gervais,M. and Terrier, R.},
   publisher = {Springer Berlin },
   isbn = {978-3-642-13594-1},
   keyword = {Computer Science},
   pages = {3-19},
   volume = {6138},
   year = {2010}
}



@article{BaBeRu2006,
	author = {Batory, D. and Benavides, D. and Ruiz-Cortes, A.},
	title = {{Automated analysis of feature models: challenges ahead}},
	journal = {Comm. of the ACM},
	volume = {49},
	issue = {12},
	year = {2006},
	issn = {0001-0782},
	pages = {45--47},
	numpages = {3},
	url = {http://doi.acm.org/10.1145/1183236.1183264},
	doi = {http://doi.acm.org/10.1145/1183236.1183264},
	acmid = {1183264},
	publisher = {ACM},
	address = {New York, NY, USA},
} 






@INPROCEEDINGS{BaPuSe2004,
	author = {Joachim Baumeister and Frank Puppe and Dietmar Seipel},
	title = {Refactoring Methods for Knowledge Bases},
	booktitle = {ENGINEERING KNOWLEDGE IN THE AGE OF THE SEMANTIC WEB: 14TH INTERNATIONAL CONFERENCE, EKAW, LNAI 3257},
	year = {2004},
	pages = {157--171},
	publisher = {Springer}
}

@INPROCEEDINGS{BaSe2007,
	author = {Joachim Baumeister and Dietmar Seipel},
	title = {Verification and Refactoring of Ontologies With Rules},
	booktitle = {In Managing Knowledge in a World of Networks: 15th International Conference, EKAW, LNAI 4248},
	year = {2007},
	publisher = {Springer}
}

@article{BaSe2010,
	author = {Baumeister, Joachim and Seipel, Dietmar},
	title = {Anomalies in ontologies with rules},
	journal = {Web Semantics: Science, Services and Agents on the World Wide Web},
	volume = {8},
	issue = {1},
	month = {March},
	year = {2010},
	issn = {1570-8268},
	pages = {55--68},
	numpages = {14},
	url = {http://dx.doi.org/10.1016/j.websem.2009.12.003},
	doi = {http://dx.doi.org/10.1016/j.websem.2009.12.003},
	acmid = {1746751},
	publisher = {Elsevier Science Publishers B. V.},
	address = {Amsterdam, The Netherlands, The Netherlands},
	keywords = {Evaluation, Ontology engineering, Owl, Rif-Bld, Swrl, Verification},
}



@INPROCEEDINGS{BeSeTrRu2006,
	author = {D. Benavides and S. Segura and P. Trinidad and A. Ruiz-Cortes},
	title = {Using Java CSP Solvers in the Automated Analyses of Feature Models},
	booktitle = {Post-Proceedings of the Summer School on Generative and Transformational Techniques in Software Engineering (GTTSE)},
	series = {LNCS},
	number = {4143},
	year = {2006},
	pages = {399 - 408}
}



@incollection{CaSuFr2005,
	author = {OCallaghan, Barry and OSullivan, Barry and Freuder, Eugene},
	editor = {van Beek, Peter},
	address = {Berlin / Heidelberg},
	booktitle = {Principles and Practice of Constraint Programming - CP 2005},
	interhash = {7ae28345c3bc813c0225a8cbed38b844},
	intrahash = {d8a977d3748ddd0ddeaed4a7f07f0c7d},
	pages = {445-459},
	publisher = {Springer},
	series = {LNCS},
	title = {Generating Corrective Explanations for Interactive Constraint Satisfaction},
	volume = 3709,
	year = 2005,
	timestamp = {2011-03-14T13:41:18.000+0100},
	keywords = {constraints correlctiverelax csp relaxation},
	added-at = {2011-03-14T13:41:18.000+0100},
	biburl = {http://www.bibsonomy.org/bibtex/2d8a977d3748ddd0ddeaed4a7f07f0c7d/mschuber}
}

@article{ChBaKu2009,
	author = {Chandola, V. and Banerjee, A. and Kumar, V.},
	title = {Anomaly detection: A survey},
	journal = {ACM Computing Surveys},
	issue_date = {July 2009},
	volume = {41},
	issue = {3},
	month = {July},
	year = {2009},
	issn = {0360-0300},
	pages = {15:1--15:58},
	articleno = {15},
	numpages = {58},
	url = {http://doi.acm.org/10.1145/1541880.1541882},
	doi = {http://doi.acm.org/10.1145/1541880.1541882},
	acmid = {1541882},
	publisher = {ACM},
	address = {New York, NY, USA},
	keywords = {Anomaly detection, outlier detection},
}

@INPROCEEDINGS{ChCoMi2006,
	author = {John Charnley and Simon Colton and Ian Miguel},
	title = {Automatic generation of implied constraints},
	booktitle = {Initial Progress, Proceedings of the 8th Workshop on Automated Reasoning},
	year = {2006},
	pages = {73--77}
}

@article{ChRa2008,
	title = "MRM: A matrix representation and mapping approach for knowledge acquisition",
	journal = "Knowledge-Based Systems",
	volume = "21",
	number = "4",
	pages = "284 - 293",
	year = "2008",
	note = "",
	issn = "0950-7051",
	doi = "10.1016/j.knosys.2007.07.002",
	url = "http://www.sciencedirect.com/science/article/pii/S0950705107000743",
	author = "Chun-Hsien Chen and Zhiming Rao",
	keywords = "Knowledge-based systems",
	keywords = "Knowledge acquisition",
	keywords = "General sorting",
	keywords = "Matrix representation and mapping",
	keywords = "Rule generation"
}

@inproceedings{FeFrJa2000,
	author = {Alexander Felfernig and Gerhard Friedrich and Dietmar Jannach and Markus Zanker},
	title = {Generating product configuration knowledge bases from precise domain extended UML models},
	year = {2000},
	booktitle = {12th International Conference on Software Engineering and Knowledge Engineering (SEKE'2000)},
	editor = {Dan Cooke},
	address = {Skokie, IL, USA},
	pages = {284--293}
}

@article{FeFrJaSt2004,
	title = "Consistency-based diagnosis of configuration knowledge bases",
	journal = "Artificial Intelligence",
	volume = "152",
	number = "2",
	pages = "213 - 234",
	year = "2004",
	note = "",
	issn = "0004-3702",
	doi = "10.1016/S0004-3702(03)00117-6",
	url = "http://www.sciencedirect.com/science/article/pii/S0004370203001176",
	author = "A. Felfernig and G. Friedrich and D. Jannach and M. Stumptner",
	keywords = "Diagnosis",
	keywords = "Configuration"
}

@inproceedings{FFSMMT2009,
	author = {Alexander Felfernig and Gerhard Friedrich and Monika Schubert and Monika Mandl and Markus Mairitsch and Erich Teppan},
	title = {Plausible Repairs for Inconsistent Requirements},
	booktitle = {IJCAI},
	year = {2009},
	pages = {791 - 796},
	ee = {http://ijcai.org/papers09/Papers/IJCAI09-136.pdf},
	crossref  = {Bo2009},
	bibsource = {DBLP, http://dblp.uni-trier.de}
}

@incollection {FeMaPuSc2010,
	author = {Felfernig, Alexander and Mandl, Monika and Pum, Anton and Schubert, Monika},
	affiliation = {Applied Software Engineering, Graz University of Technology, Inffeldgasse 16b, A-8010 Graz, Austria},
	title = {Empirical Knowledge Engineering: Cognitive Aspects in the Development of Constraint-Based Recommenders},
	booktitle = {Trends in Applied Intelligent Systems},
	series = {LNCS},
	editor = {Garcï¿½a-Pedrajas, Nicolï¿½s and Herrera, Francisco and Fyfe, Colin and Benï¿½tez, Josï¿½ and Ali, Moonis},
	publisher = {Springer Berlin / Heidelberg},
	isbn = {978-3-642-13021-2},
	keyword = {Computer Science},
	pages = {631-640},
	volume = {6096},
	year = {2010}
}

@article{FeSc2011,
	author = {Alexander Felfernig and Monika Schubert},
	title = {Personalized diagnoses for inconsistent user requirements},
	journal = {AI EDAM},
	volume = {25},
	number = {2},
	year = {2011},
	pages = {175-183},
	ee = {http://dx.doi.org/10.1017/S0890060410000612},
	bibsource = {DBLP, http://dblp.uni-trier.de}
}

@inproceedings{FeZeBl2010,
	author = {Alexander Felfernig and Christoph Zehentner and Paul Blazek},
	editor = {Martin Sachenbacher and Oskar Dressler and Michael Hofbaur},
	title = {CoreDiag: Eliminating Redundancy in Constraint Sets},
	booktitle = {DX 2011. 22nd International Workshop on Principles of Diagnosis},
	year = {2010},
	pages = {219 - 224},
	address = {Murnau, GER},
}

@inproceedings{FSMFT2010,
	author    = {Alexander Felfernig and Monika Schubert and Monika Mandl and Gerhard Friedrich and Erich Teppan},
	title = {Efficient Explanations for Inconsistent Constraint Sets},
	booktitle = {ECAI},
	year = {2010},
	pages = {1043-1044},
	ee = {http://dx.doi.org/10.3233/978-1-60750-606-5-1043},
	crossref = {CoStWo2010},
	bibsource = {DBLP, http://dblp.uni-trier.de}
}

@INPROCEEDINGS{GiTa1997,
	author = {Yolanda Gil and A Gil and Marcelo Tallis},
	title = {A Script-Based Approach to Modifying Knowledge Bases},
	booktitle = {In Proceedings of the Fourteenth National Conference on Artificial Intelligence and Ninth Innovative Application of Artificial Intelligence Conference AAAI},
	year = {1997},
	pages = {377--383},
	publisher = {Press/ MIT Press}
}

@article{GoLeLe2003,
	author = {Cleotilde Gonzalez and Javier F. Lerch and Christian Lebiere},
	year = {2003},
	title = {Instance-based learning in dynamic decision making},
	journal = {Cognitive Science},
	volume = {27},
	number = {4},
	pages = {591 - 635}
}

@article{GrHu2006,
	author = {Grant, John and Hunter, Anthony},
	title = {Measuring inconsistency in knowledgebases},
	journal = {Journal of Intelligent Information Systems},
	volume = {27},
	issue = {2},
	month = {September},
	year = {2006},
	issn = {0925-9902},
	pages = {159--184},
	numpages = {26},
	url = {http://dl.acm.org/citation.cfm?id=1164494.1164503},
	doi = {10.1007/s10844-006-2974-4},
	acmid = {1164503},
	publisher = {Kluwer Academic Publishers},
	address = {Hingham, MA, USA},
	keywords = {Conflict resolution, Inconsistency, Integrity constraints, Paraconsistent logic},
}

@INPROCEEDINGS{GuPuFrWa1996,
	author = {Jun Gu and Paul W. Purdom and John Franco and Benjamin W. Wah},
	title = {Algorithms for the Satisfiability (SAT) Problem: A Survey},
	booktitle = {DIMACS Series in Discrete Mathematics and Theoretical Computer Science},
	year = {1996},
	pages = {19--152},
	publisher = {American Mathematical Society}
}

@article{HaJoFa1993,
	author = {Bruce G. S. Hardie and Eric J. Johnson and Peter S. Fader},
	year = {1993},
	title = {Modeling Loss Aversion and Reference Dependence Effects on Brand Choice},
	journal = {Marketing Science},
	volume = {12},
	number = {4},
	pages = {378 - 394}
}

@inproceedings{HaTr2008,
	author = {Hartmann, Herman and Trew, Tim},
	title = {Using Feature Diagrams with Context Variability to Model Multiple Product Lines for Software Supply Chains},
	booktitle = {Proceedings of the 2008 12th International Software Product Line Conference},
	year = {2008},
	isbn = {978-0-7695-3303-2},
	pages = {12--21},
	numpages = {10},
	url = {http://dl.acm.org/citation.cfm?id=1439277.1440007},
	doi = {10.1109/SPLC.2008.15},
	acmid = {1440007},
	publisher = {IEEE Computer Society},
	address = {Washington, DC, USA},
	keywords = {Feature Modelling, Variability, Multiple Product Lines, Software Supply Chains, Staged Configuration},
} 

@inproceedings{He2008,
	title = {Finding Contradictions in Feature Models},
	author = {Adithya Hemakumar},
	year = {2008},
	researchr = {http://researchr.org/publication/Hemakumar08},
	cites = {0},
	citedby = {0},
	pages = {183-190},
	booktitle = {Software Product Lines, 12th International Conference, SPLC 2008, Limerick, Ireland, September 8-12, 2008, Proceedings. Second Volume (Workshops)},
	editor = {Steffen Thiel and Klaus Pohl},
	publisher = {Lero Int. Science Centre, University of Limerick, Ireland},
	isbn = {978-1-905952-06-9},
}

@article{HeKoTeRi2004,
	author = {Herlocker, Jonathan L. and Konstan, Joseph A. and Terveen, Loren G. and Riedl, John T.},
	title = {Evaluating collaborative filtering recommender systems},
	journal = {ACM Trans. Inf. Syst.},
	volume = {22},
	issue = {1},
	month = {January},
	year = {2004},
	issn = {1046-8188},
	pages = {5--53},
	numpages = {49},
	url = {http://doi.acm.org/10.1145/963770.963772},
	doi = {http://doi.acm.org/10.1145/963770.963772},
	acmid = {963772},
	publisher = {ACM},
	address = {New York, NY, USA},
	keywords = {Collaborative filtering, evaluation, metrics, recommender systems},
} 

@article{HuCh2008,
	author = {Huang, Chin-Jung and Cheng, Min-Yuan},
	title = {Conflicting treatment model for certainty rule-based knowledge},
	journal = {Expert Systems with Applications},
	volume = {35},
	issue = {1-2},
	month = {July},
	year = {2008},
	issn = {0957-4174},
	pages = {161--176},
	numpages = {16},
	url = {http://dl.acm.org/citation.cfm?id=1379458.1379545},
	doi = {10.1016/j.eswa.2007.06.015},
	acmid = {1379545},
	publisher = {Pergamon Press, Inc.},
	address = {Tarrytown, NY, USA},
	keywords = {Conditional probability, Group decision, Knowledge similarity, Reliability factor, Weighted average theory},
}

@inproceedings{Junker2004,
	author = {U. Junker},
	booktitle = {19th National Conference on Artifical Intelligence},
	pages = {167--172},
	publisher = {AAAI},
	series = {AAAI 2004},
	title = {\textsc{QuickXPlain}: {preferred explanations and relaxations for over-constrained problems}},
	year = 2004
}


@inproceedings{Acher2018,
	author = {M. Acher and P. Temple and J-M. Jézéquel and J. Galindo and J. Martinez and T. Tiadi},
	booktitle = {12th International Workshop on Variability Modelling of Software-Intensive Systems},
	pages = {83--88},
	title = {\textsc{VaryLaTeX}: {Learning Paper Variants That Meet Constraints}},
	address = {Madrid, Spain},
	year = 2018
}






@techreport{KCHNP1990,
	author = {K. Kang and S. Cohen and J. Hess and W. Novak and A. Peterson},
	institution = {Carnegie-Mellon University Software Engineering Institute},
	title = {Feature-Oriented Domain Analysis (FODA) Feasibility Study},
	year = 1990, 
	publisher = {{Carnegie Mellon University Technical Report SEI-90-TR-21}}
}

@inproceedings{Kl1990,
	author = {De Kleer, Johan},
	title = {A comparison of ATMS and CSP techniques},
	booktitle = {Proceedings of the 11th international joint conference on Artificial intelligence - Volume 1},
	year = {1989},
	location = {Detroit, Michigan},
	pages = {290--296},
	numpages = {7},
	url = {http://dl.acm.org/citation.cfm?id=1623755.1623800},
	acmid = {1623800},
	publisher = {Morgan Kaufmann Publishers Inc.},
	address = {San Francisco, CA, USA},
}

@inproceedings{KoSc2000,
	author = {Ivan Koychev and Ingo Schwab},
	title = {Adaptation to Drifting User's Interests},
	booktitle = {In Proceedings of ECML2000 Workshop: Machine Learning in New Information Age},
	year = {2000},
	pages = {39 - 46}
}



@incollection {Ma2002,
   author = {Mannion, Mike},
   affiliation = {Glasgow Caledonian University School of Computing and Mathematical Sciences 70 Cowcaddens Road Glasgow G4 0BA Scotland, UK},
   title = {Using First-Order Logic for Product Line Model Validation},
   booktitle = {Software Product Lines},
   series = {LNCS},
   editor = {Chastek, Gary},
   publisher = {Springer Berlin / Heidelberg},
   isbn = {978-3-540-43985-1},
   keyword = {Computer Science},
   pages = {149-202},
   volume = {2379},
   year = {2002}
}



@article{MeCo2010,
	title = {{Decision-making coordination and efficient reasoning techniques for feature-based configuration}},
	journal = "Science of Computer Programming",
	volume = "75",
	number = "5",
	pages = "311 - 332",
	year = "2010",
	note = "Coordination Models, Languages and Applications(SAC 2008)",
	issn = "0167-6423",
	doi = "10.1016/j.scico.2009.12.004",
	url = "http://www.sciencedirect.com/science/article/pii/S0167642309001713",
	author = "M. Mendonca and D. Cowan",
	keywords = "Software Product Lines",
	keywords = "Product configuration",
	keywords = "Feature models",
	keywords = "Feature modelling",
	keywords = "Decision-making coordination",
	keywords = "Automated reasoning",
	keywords = "Constraint-based reasoning"
}

@techreport{OCL,
	author = {{Object Management Group (OMG)}},
	title = {Object Constraint Language v2.0},
	year = {2006}
}

@inproceedings{Pi2008,
	author = {Piette, C\'{e}dric},
	title = {Let the Solver Deal with Redundancy},
	booktitle = {Proceedings of the 2008 20th IEEE International Conference on Tools with Artificial Intelligence - Volume 01},
	year = {2008},
	isbn = {978-0-7695-3440-4},
	pages = {67--73},
	numpages = {7},
	url = {http://dl.acm.org/citation.cfm?id=1474547.1474573},
	doi = {10.1109/ICTAI.2008.38},
	acmid = {1474573},
	publisher = {IEEE Computer Society},
	address = {Washington, DC, USA},
	keywords = {SAT, redundancy management},
} 



@ARTICLE{PrSh1994,
	author = {Alun D. Preece and Rajjan Shinghal},
	title = {Foundation and application of knowledge base verification},
	journal = {International Journal of Intelligent Systems 1994;9(8):683ï¿½702. Duftschmid, S. Miksch / },
	year = {1994},
	volume = {22},
	pages = {23--41}
}

@article{Pr1987,
	author = {David S. Prerau},
	title = {Knowledge Acquisition in the Development of a Large Expert System},
	journal = {AI Magazine},
	volume = {8},
	number = {2},
	year = {1987},
	pages = {43-51},
	ee = {http://www.aaai.org/ojs/index.php/aimagazine/article/view/582},
	bibsource = {DBLP, http://dblp.uni-trier.de}
}



@inbook{RFRJD2008,
	title={Knowledge Patterns and Knowledge Refactorings for increasing the Quality of Knowledge},
	number={17},
	series = {premier reference source},
	booktitle={Knowledge Management Organizational Memory and Transfer Behavior Global Approaches and Advancements},
	publisher={IGI},
	author={Rech, Jï¿½rg and Feldmann, Raimund L. and Ras, Eric and Jedlitschka, Andreas and Decker, Bjï¿½rn and Jennex, Murray E.},
	year={2008},
	pages={281--328}
}

@article{RSHYD2005,
	author = {Reiter, Ehud and Sripada, Somayajulu and Hunter, Jim and Yu, Jin and Davy, Ian},
	title = {Choosing words in computer-generated weather forecasts},
	journal = {Artificial Intelligence},
	volume = {167},
	issue = {1-2},
	month = {September},
	year = {2005},
	issn = {0004-3702},
	pages = {137--169},
	numpages = {33},
	url = {http://dl.acm.org/citation.cfm?id=1113166.1644548},
	doi = {10.1016/j.artint.2005.06.006},
	acmid = {1644548},
	publisher = {Elsevier Science Publishers Ltd.},
	address = {Essex, UK},
	keywords = {Idiolect, Information presentation, Language and the word, Lexical choice, Natural language generation, Natural language processing, Weather forecasts},
}

@book{RuNo2003,
	author = {Stuart Russell and Peter Norvig},
	title = {Artificial Intelligence. A modern approach},
	volume = {2},
	year = {2003},
	publisher = {Prentice Hall},
	address = {New Jersey}
}



@book{jannach2010,
	author = {D. Jannach and M. Zanker and A. Felfernig and G. Friedrich},
	title = {Recommender Systems -- An Introduction},
	year = {2010},
	publisher = {Cambridge University Press}
}



@inproceedings{ScFe2010,
    author = {Schubert, Monika and Felfernig, Alexander},
    booktitle = {Proceedings of the 21st International Workshop on the Principles of Diagnosis},
    citeulike-article-id = {8502444},
    keywords = {cite-ijcai11, diagnosis, diagnosis\_algorithm, fastdiag},
    posted-at = {2011-01-04 10:04:39},
    priority = {2},
    title = {{A Diagnosis Algorithm for Inconsistent Constraint Sets}},
    year = {2010}
}

@inproceedings{ScFeMa2010,
   author = {Schubert, Monika and Felfernig, Alexander and Mandl, Monika},
   affiliation = {Applied Software Engineering, IST, Graz University of Technology, Inffeldgasse 16b/II, 8010 Graz, Austria},
   title = {FastXplain: Conflict Detection for Constraint-Based Recommendation Problems},
   booktitle = {Trends in Applied Intelligent Systems},
   series = {LNCS},
   editor = {Garcia-Pedrajas, Nicolas and Herrera, Francisco and Fyfe, Colin and Benï¿½tez, Josï¿½ and Ali, Moonis},
   publisher = {Springer Berlin / Heidelberg},
   isbn = {978-3-642-13021-2},
   keyword = {Computer Science},
   pages = {621-630},
   volume = {6096},
   year = {2010}
}

@INPROCEEDINGS{ScHeTr2006,
	author={Schobbens, P.-Y. and Heymans, P. and Trigaux, J.-C.},
	booktitle={Requirements Engineering, 14th IEEE International Conference},
	title={Feature Diagrams: A Survey and a Formal Semantics},
	year={2006},
	month={sept.},
	volume={},
	number={},
	pages={139 -148},
	keywords={engineering requirements;feature oriented domain analysis method;formal semantics;free feature diagrams;modelling languages;software product lines;formal specification;formal verification;},
	doi={10.1109/RE.2006.23},
	ISSN={1090-705X}
}

@INPROCEEDINGS{SGBPR2012,
  author = {S. Segura and J.A. Galindo and D. Benavides and J.A. Parejo and A. Ruiz-Cortes},
  title = {BeTTy: Benchmarking and Testing on the Automated Analysis of Feature Models},
  booktitle = {6th International Workshop on Variability Modelling of Software-intensive Systems (VaMoS'12)},
  year = {2012},
  editor = {U. Eisenecker and S. Apel and S. Gnesi},
  pages = {63--71},
  address = {Leipzig, Germany},
  publisher = {ACM}
}

@INPROCEEDINGS{SeHiBeRu2010,
	author={S. Segura and R. Hierons and D. Benavides and A. Ruiz-Cortes},
	booktitle={3rd Intl. Conference on Software Testing, Verification and Validation (ICST)},
	title={Automated Test Data Generation on the Analyses of Feature Models: A Metamorphic Testing Approach},
	year={2010},
	pages={35--44}
}

@article{SeHiBeRu2011,
	title = "Automated metamorphic testing on the analyses of feature models",
	journal = "Information and Software Technology",
	volume = "53",
	number = "3",
	pages = "245 - 258",
	year = "2011",
	author = "S. Segura and R. Hierons and D. Benavides and A. Ruiz-Cortes"
}

@article{Sp2006,
	title = "The influence of information presentation formats on complex task decision-making performance",
	journal = "International Journal of Human-Computer Studies",
	volume = "64",
	number = "11",
	pages = "1115 - 1131",
	year = "2006",
	note = "",
	issn = "1071-5819",
	doi = "10.1016/j.ijhcs.2006.06.007",
	url = "http://www.sciencedirect.com/science/article/pii/S1071581906001042",
	author = "Cheri and Speier",
	keywords = "Complex task",
	keywords = "Decision-making",
	keywords = "Information presentation"
}


@INPROCEEDINGS{SJHAHM2004,
    author = {Sathiamoorthy Subbarayan and Rune M. Jensen and Tarik Hadzic and Henrik R. Andersen and Henrik Hulgaard and Jesper Moeller},
    title = {Comparing Two Implementations of a Complete and Backtrack-Free Interactive Configurator},
    booktitle = {In: CP 2004 CSPIA Workshop},
    year = {2004},
    pages = {97--111}
}




@incollection {WXHZZM2010,
	author = {Wang, B. and Xiong, Y. and Hu, Z. and Zhao, H. and Zhang, W. and Mei, H.},
	affiliation = {Key Laboratory of High Confidence Software Technologies, Ministry of Education, China},
	title = {A Dynamic-Priority Based Approach to Fixing Inconsistent Feature Models},
	booktitle = {Model Driven Engineering Languages and Systems},
	series = {LNCS},
	editor = {D. Petriu and N. Rouquette and O. Haugen},
	publisher = {Springer Berlin},
	pages = {181--195},
	volume = {6394},
	year = {2010}
}

@inproceedings{Zh2005,
	author = {Zhang, Du},
	title = {Fixpoint semantics for rule-base anomalies},
	booktitle = {Proceedings of the Fourth IEEE International Conference on Cognitive Informatics},
	year = {2005},
	isbn = {0-7803-9136-5},
	pages = {10--17},
	numpages = {8},
	url = {http://dl.acm.org/citation.cfm?id=1259589.1259932},
	acmid = {1259932},
	publisher = {IEEE Computer Society},
	address = {Washington, DC, USA},
}

@incollection{Zh2008,
	author = {Zhang, Du},
	chapter = {Quantifying knowledge base inconsistency via fixpoint semantics},
	title = {Transactions on computational science II},
	booktitle = {LNCS},
	number = {5150},
	editor = {Gavrilova, Marina L. and Tan, C. J. Kenneth and Wang, Yingxu and Yao, Yiyu and Wang, Guoyin},
	year = {2008},
	isbn = {3-540-87562-X, 978-3-540-87562-8},
	pages = {145--160},
	numpages = {16},
	url = {http://dl.acm.org/citation.cfm?id=1806222.1806232},
	acmid = {1806232},
	publisher = {Springer-Verlag},
	address = {Berlin, Heidelberg},
	keywords = {KB coherence, fixpoint semantics, inconsistency, significance of inconsistency},
}

@article{ZhNg1994,
	author = {Du Zhang and Doan Nguyen},
	title = {PREPARE: A Tool for Knowledge Base Verification},
	year = {1994},
	volume = {6},
	number = {6},
	pages = {983--989},
	journal = {IEEE Transactions on Knowledge and Data Engineering}
}

@incollection {ZhZh2008,
	author = {Zhang, Du and Zhu, Hong},
	affiliation = {California State University Department of Computer Science Sacramento CA 95819-6021 USA},
	title = {When Is Inconsistency Considered Harmful: Temporal Characterization of Knowledge Base Inconsistency},
	booktitle = {Intelligent Data Engineering and Automated Learning, IDEAL 2008},
	series = {LNCS},
	editor = {Fyfe, Colin and Kim, Dongsup and Lee, Soo-Young and Yin, Hujun},
	publisher = {Springer Berlin / Heidelberg},
	isbn = {978-3-540-88905-2},
	keyword = {Computer Science},
	pages = {395-403},
	volume = {5326},
	year = {2008}
}



main.bbl
%%% -*-BibTeX-*-
%%% Do NOT edit. File created by BibTeX with style
%%% ACM-Reference-Format-Journals [18-Jan-2012].

\begin{thebibliography}{34}

%%% ====================================================================
%%% NOTE TO THE USER: you can override these defaults by providing
%%% customized versions of any of these macros before the \bibliography
%%% command.  Each of them MUST provide its own final punctuation,
%%% except for \shownote{}, \showDOI{}, and \showURL{}.  The latter two
%%% do not use final punctuation, in order to avoid confusing it with
%%% the Web address.
%%%
%%% To suppress output of a particular field, define its macro to expand
%%% to an empty string, or better, \unskip, like this:
%%%
%%% \newcommand{\showDOI}[1]{\unskip}   % LaTeX syntax
%%%
%%% \def \showDOI #1{\unskip}           % plain TeX syntax
%%%
%%% ====================================================================

\ifx \showCODEN    \undefined \def \showCODEN     #1{\unskip}     \fi
\ifx \showDOI      \undefined \def \showDOI       #1{#1}\fi
\ifx \showISBNx    \undefined \def \showISBNx     #1{\unskip}     \fi
\ifx \showISBNxiii \undefined \def \showISBNxiii  #1{\unskip}     \fi
\ifx \showISSN     \undefined \def \showISSN      #1{\unskip}     \fi
\ifx \showLCCN     \undefined \def \showLCCN      #1{\unskip}     \fi
\ifx \shownote     \undefined \def \shownote      #1{#1}          \fi
\ifx \showarticletitle \undefined \def \showarticletitle #1{#1}   \fi
\ifx \showURL      \undefined \def \showURL       {\relax}        \fi
% The following commands are used for tagged output and should be
% invisible to TeX
\providecommand\bibfield[2]{#2}
\providecommand\bibinfo[2]{#2}
\providecommand\natexlab[1]{#1}
\providecommand\showeprint[2][]{arXiv:#2}

\bibitem[\protect\citeauthoryear{Acher, Temple, Jézéquel, Galindo, Martinez,
  and Tiadi}{Acher et~al\mbox{.}}{2018}]%
        {Acher2018}
\bibfield{author}{\bibinfo{person}{M. Acher}, \bibinfo{person}{P. Temple},
  \bibinfo{person}{J-M. Jézéquel}, \bibinfo{person}{J. Galindo},
  \bibinfo{person}{J. Martinez}, {and} \bibinfo{person}{T. Tiadi}.}
  \bibinfo{year}{2018}\natexlab{}.
\newblock \showarticletitle{\textsc{VaryLaTeX}: {Learning Paper Variants That
  Meet Constraints}}. In \bibinfo{booktitle}{\emph{12th International Workshop
  on Variability Modelling of Software-Intensive Systems}}.
  \bibinfo{address}{Madrid, Spain}, \bibinfo{pages}{83--88}.
\newblock


\bibitem[\protect\citeauthoryear{Apel and Kästner}{Apel and Kästner}{2009}]%
        {Apel2009}
\bibfield{author}{\bibinfo{person}{S. Apel} {and} \bibinfo{person}{C.
  Kästner}.} \bibinfo{year}{2009}\natexlab{}.
\newblock \showarticletitle{An Overview of Feature-Oriented Software
  Development}.
\newblock \bibinfo{journal}{\emph{Journal of Object Technology}}
  \bibinfo{volume}{8}, \bibinfo{number}{5} (\bibinfo{year}{2009}),
  \bibinfo{pages}{49--84}.
\newblock


\bibitem[\protect\citeauthoryear{Benavides, Segura, and Ruiz-Cortes}{Benavides
  et~al\mbox{.}}{2010}]%
        {BeSeRu2010}
\bibfield{author}{\bibinfo{person}{D. Benavides}, \bibinfo{person}{S. Segura},
  {and} \bibinfo{person}{A. Ruiz-Cortes}.} \bibinfo{year}{2010}\natexlab{}.
\newblock \showarticletitle{{Automated analysis of feature models 20 years
  later: A literature review}}.
\newblock \bibinfo{journal}{\emph{Information Systems}}  \bibinfo{volume}{35}
  (\bibinfo{year}{2010}), \bibinfo{pages}{615--636}.
\newblock
Issue 6.


\bibitem[\protect\citeauthoryear{Burke}{Burke}{2000}]%
        {Burke2000}
\bibfield{author}{\bibinfo{person}{R. Burke}.} \bibinfo{year}{2000}\natexlab{}.
\newblock \showarticletitle{{Knowledge-based recommender systems}}.
\newblock \bibinfo{journal}{\emph{Encyclopedia of Library and Information
  Systems}} \bibinfo{volume}{69}, \bibinfo{number}{32} (\bibinfo{year}{2000}),
  \bibinfo{pages}{180--200}.
\newblock


\bibitem[\protect\citeauthoryear{Bécan, Behjati, Gotlieb, and Acher}{Bécan
  et~al\mbox{.}}{2015}]%
        {Becan2015}
\bibfield{author}{\bibinfo{person}{G. Bécan}, \bibinfo{person}{R. Behjati},
  \bibinfo{person}{A. Gotlieb}, {and} \bibinfo{person}{M. Acher}.}
  \bibinfo{year}{2015}\natexlab{}.
\newblock \showarticletitle{{Synthesis of attributed feature models from
  product descriptions}}. In \bibinfo{booktitle}{\emph{{19th International
  Software Product Line Conference}}}. \bibinfo{address}{Nashville, TN, USA},
  \bibinfo{pages}{1--10}.
\newblock


\bibitem[\protect\citeauthoryear{Ekstrand, Riedl, and Konstan}{Ekstrand
  et~al\mbox{.}}{2011}]%
        {Ekstrand2011}
\bibfield{author}{\bibinfo{person}{M. Ekstrand}, \bibinfo{person}{J. Riedl},
  {and} \bibinfo{person}{J. Konstan}.} \bibinfo{year}{2011}\natexlab{}.
\newblock \bibinfo{booktitle}{\emph{{Collaborative Filtering Recommender
  Systems}}}. \bibinfo{series}{{Foundations and Trends in Human–Computer
  Interaction}}, Vol.~\bibinfo{volume}{4}.
\newblock


\bibitem[\protect\citeauthoryear{Falkner, Felfernig, and Haag}{Falkner
  et~al\mbox{.}}{2011}]%
        {Falkner2011}
\bibfield{author}{\bibinfo{person}{A. Falkner}, \bibinfo{person}{A. Felfernig},
  {and} \bibinfo{person}{A. Haag}.} \bibinfo{year}{2011}\natexlab{}.
\newblock \showarticletitle{{Recommendation Technologies for Configurable
  Products}}.
\newblock \bibinfo{journal}{\emph{AI Magazine}} \bibinfo{volume}{32},
  \bibinfo{number}{3} (\bibinfo{year}{2011}), \bibinfo{pages}{99--108}.
\newblock


\bibitem[\protect\citeauthoryear{Felfernig, Atas, Tran, and
  Stettinger}{Felfernig et~al\mbox{.}}{2016}]%
        {Felfernigetal2016}
\bibfield{author}{\bibinfo{person}{A. Felfernig}, \bibinfo{person}{M. Atas},
  \bibinfo{person}{T. Tran}, {and} \bibinfo{person}{M. Stettinger}.}
  \bibinfo{year}{2016}\natexlab{}.
\newblock \showarticletitle{Towards Group-based Configuration}. In
  \bibinfo{booktitle}{\emph{Workshop on Configuration (ConfWS'16)}}.
  \bibinfo{address}{Toulouse, France}, \bibinfo{pages}{69--72}.
\newblock


\bibitem[\protect\citeauthoryear{Felfernig, Boratto, Stettinger, and
  Tkalcic}{Felfernig et~al\mbox{.}}{2018}]%
        {Felfernigetal2018}
\bibfield{author}{\bibinfo{person}{A. Felfernig}, \bibinfo{person}{L. Boratto},
  \bibinfo{person}{M. Stettinger}, {and} \bibinfo{person}{M. Tkalcic}.}
  \bibinfo{year}{2018}\natexlab{}.
\newblock \bibinfo{booktitle}{\emph{Group Recommender Systems}}.
\newblock \bibinfo{publisher}{Springer}.
\newblock


\bibitem[\protect\citeauthoryear{Felfernig and Burke}{Felfernig and
  Burke}{2008}]%
        {FelfernigBurke2008}
\bibfield{author}{\bibinfo{person}{A. Felfernig} {and} \bibinfo{person}{R.
  Burke}.} \bibinfo{year}{2008}\natexlab{}.
\newblock \showarticletitle{{Constraint-based Recommender Systems: Technologies
  and Research Issues}}. In \bibinfo{booktitle}{\emph{ACM International
  Conference on Electronic Commerce}}. \bibinfo{address}{Innsbruck, Austria},
  \bibinfo{pages}{17--26}.
\newblock


\bibitem[\protect\citeauthoryear{Felfernig, Friedrich, Jannach, and
  Zanker}{Felfernig et~al\mbox{.}}{2006}]%
        {Felfernig2006}
\bibfield{author}{\bibinfo{person}{A. Felfernig}, \bibinfo{person}{G.
  Friedrich}, \bibinfo{person}{D. Jannach}, {and} \bibinfo{person}{M. Zanker}.}
  \bibinfo{year}{2006}\natexlab{}.
\newblock \showarticletitle{{An Integrated Environment for the Development of
  Knowledge-based Recommender Applications}}.
\newblock \bibinfo{journal}{\emph{Intl. Journal of Electronic Commerce}}
  \bibinfo{volume}{11}, \bibinfo{number}{2} (\bibinfo{year}{2006}),
  \bibinfo{pages}{11--34}.
\newblock


\bibitem[\protect\citeauthoryear{Felfernig, Hotz, Bagley, and
  Tiihonen}{Felfernig et~al\mbox{.}}{2014}]%
        {Felfernigetal2014}
\bibfield{author}{\bibinfo{person}{A. Felfernig}, \bibinfo{person}{L. Hotz},
  \bibinfo{person}{C. Bagley}, {and} \bibinfo{person}{J. Tiihonen}.}
  \bibinfo{year}{2014}\natexlab{}.
\newblock \bibinfo{booktitle}{\emph{Knowledge-based Configuration - From
  Research to Business Cases}}.
\newblock \bibinfo{publisher}{Elsevier}.
\newblock


\bibitem[\protect\citeauthoryear{Felfernig, Reiterer, Stettinger, Reinfrank,
  Jeran, and Ninaus}{Felfernig et~al\mbox{.}}{2013}]%
        {Felfernigetal2013}
\bibfield{author}{\bibinfo{person}{A. Felfernig}, \bibinfo{person}{S.
  Reiterer}, \bibinfo{person}{M. Stettinger}, \bibinfo{person}{F. Reinfrank},
  \bibinfo{person}{M. Jeran}, {and} \bibinfo{person}{G. Ninaus}.}
  \bibinfo{year}{2013}\natexlab{}.
\newblock \showarticletitle{Recommender Systems for Configuration Knowledge
  Engineering}. In \bibinfo{booktitle}{\emph{Workshop on Configuration
  (ConfWS'13)}}. \bibinfo{address}{Vienna, Austria}, \bibinfo{pages}{51--54}.
\newblock


\bibitem[\protect\citeauthoryear{Felfernig, Schubert, and Zehentner}{Felfernig
  et~al\mbox{.}}{2012}]%
        {Felfernig2012}
\bibfield{author}{\bibinfo{person}{A. Felfernig}, \bibinfo{person}{M.
  Schubert}, {and} \bibinfo{person}{C. Zehentner}.}
  \bibinfo{year}{2012}\natexlab{}.
\newblock \showarticletitle{An Efficient Diagnosis Algorithm for Inconsistent
  Constraint Sets}.
\newblock \bibinfo{journal}{\emph{AI for Engineering Design, Analysis, and
  Manufacturing (AIEDAM)}} \bibinfo{volume}{26}, \bibinfo{number}{1}
  (\bibinfo{year}{2012}), \bibinfo{pages}{53--62}.
\newblock


\bibitem[\protect\citeauthoryear{Janota, Botterweck, and Marques-Silva}{Janota
  et~al\mbox{.}}{2014}]%
        {Janota2014}
\bibfield{author}{\bibinfo{person}{M. Janota}, \bibinfo{person}{G. Botterweck},
  {and} \bibinfo{person}{J. Marques-Silva}.} \bibinfo{year}{2014}\natexlab{}.
\newblock \showarticletitle{On lazy and eager interactive reconfiguration}. In
  \bibinfo{booktitle}{\emph{8th International Workshop on Variability Modelling
  of Software-intensive Systems (VaMoS'14)}}. \bibinfo{publisher}{ACM},
  \bibinfo{address}{Sophia Antipolis, France}, \bibinfo{pages}{8:1--8:8}.
\newblock


\bibitem[\protect\citeauthoryear{Junker}{Junker}{2004}]%
        {Junker2004}
\bibfield{author}{\bibinfo{person}{U. Junker}.}
  \bibinfo{year}{2004}\natexlab{}.
\newblock \showarticletitle{\textsc{QuickXPlain}: {preferred explanations and
  relaxations for over-constrained problems}}. In
  \bibinfo{booktitle}{\emph{19th National Conference on Artifical
  Intelligence}} \emph{(\bibinfo{series}{AAAI 2004})}.
  \bibinfo{publisher}{AAAI}, \bibinfo{pages}{167--172}.
\newblock


\bibitem[\protect\citeauthoryear{Kang, Cohen, Hess, Novak, and Peterson}{Kang
  et~al\mbox{.}}{1990}]%
        {Kang1990}
\bibfield{author}{\bibinfo{person}{K. Kang}, \bibinfo{person}{S. Cohen},
  \bibinfo{person}{J. Hess}, \bibinfo{person}{W. Novak}, {and}
  \bibinfo{person}{S. Peterson}.} \bibinfo{year}{1990}\natexlab{}.
\newblock \showarticletitle{{Feature-oriented Domain Analysis (FODA) --
  Feasibility Study}}.
\newblock \bibinfo{journal}{\emph{TechnicalReport CMU -- SEI-90-TR-21}}
  (\bibinfo{year}{1990}).
\newblock


\bibitem[\protect\citeauthoryear{Konstan, Miller, Herlocker, Gordon, and
  Riedl}{Konstan et~al\mbox{.}}{1997}]%
        {Konstan1997}
\bibfield{author}{\bibinfo{person}{J. Konstan}, \bibinfo{person}{B. Miller},
  \bibinfo{person}{J. Herlocker}, \bibinfo{person}{L. Gordon}, {and}
  \bibinfo{person}{J. Riedl}.} \bibinfo{year}{1997}\natexlab{}.
\newblock \showarticletitle{{GroupLens: Applying Collaborative Filtering to
  Usenet News}}.
\newblock \bibinfo{journal}{\emph{Commun. ACM}} \bibinfo{volume}{40},
  \bibinfo{number}{3} (\bibinfo{year}{1997}), \bibinfo{pages}{77--87}.
\newblock


\bibitem[\protect\citeauthoryear{Koren, Bell, and Volinsky}{Koren
  et~al\mbox{.}}{2009}]%
        {Koren2009}
\bibfield{author}{\bibinfo{person}{Y. Koren}, \bibinfo{person}{R. Bell}, {and}
  \bibinfo{person}{C. Volinsky}.} \bibinfo{year}{2009}\natexlab{}.
\newblock \showarticletitle{{Matrix factorization techniques for recommender
  systems}}.
\newblock \bibinfo{journal}{\emph{IEEE Computer}}  \bibinfo{volume}{42}
  (\bibinfo{year}{2009}), \bibinfo{pages}{30--37}.
\newblock
Issue 8.


\bibitem[\protect\citeauthoryear{Pazzani and Billsus}{Pazzani and
  Billsus}{1997}]%
        {Pazzani1997}
\bibfield{author}{\bibinfo{person}{M. Pazzani} {and} \bibinfo{person}{D.
  Billsus}.} \bibinfo{year}{1997}\natexlab{}.
\newblock \showarticletitle{{Learning and revising user profiles: The
  identification of interesting web sites}}.
\newblock \bibinfo{journal}{\emph{Machine Learning}}  \bibinfo{volume}{27}
  (\bibinfo{year}{1997}), \bibinfo{pages}{313--331}.
\newblock


\bibitem[\protect\citeauthoryear{Pereira, Martin, Acher, Jézéquel,
  Botterweck, and Ventresque}{Pereira et~al\mbox{.}}{2019}]%
        {Pereira2020spaces}
\bibfield{author}{\bibinfo{person}{J. Pereira}, \bibinfo{person}{H. Martin},
  \bibinfo{person}{M. Acher}, \bibinfo{person}{J-M. Jézéquel},
  \bibinfo{person}{G. Botterweck}, {and} \bibinfo{person}{A. Ventresque}.}
  \bibinfo{year}{2019}\natexlab{}.
\newblock \showarticletitle{Learning Software Configuration Spaces: {A}
  Systematic Literature Review}.
\newblock \bibinfo{journal}{\emph{CoRR}}  \bibinfo{volume}{abs/1906.03018}.
\newblock


\bibitem[\protect\citeauthoryear{Pereira, Martin, Temple, and Acher}{Pereira
  et~al\mbox{.}}{2020}]%
        {Pereira2020}
\bibfield{author}{\bibinfo{person}{J. Pereira}, \bibinfo{person}{H. Martin},
  \bibinfo{person}{P. Temple}, {and} \bibinfo{person}{M. Acher}.}
  \bibinfo{year}{2020}\natexlab{}.
\newblock \showarticletitle{{Machine Learning and Configurable Systems: A
  Gentle Introduction}}. In \bibinfo{booktitle}{\emph{{24th ACM Systems and
  Software Product Line Conference}}} \emph{(\bibinfo{series}{SPLC},
  \bibinfo{number}{40})}. \bibinfo{publisher}{ACM}, \bibinfo{pages}{Vol. A}.
\newblock


\bibitem[\protect\citeauthoryear{Pereira, Matuszyk, Krieter, Spiliopoulou, and
  Saake}{Pereira et~al\mbox{.}}{2018}]%
        {Pereira2018}
\bibfield{author}{\bibinfo{person}{J. Pereira}, \bibinfo{person}{P. Matuszyk},
  \bibinfo{person}{S. Krieter}, \bibinfo{person}{M. Spiliopoulou}, {and}
  \bibinfo{person}{G. Saake}.} \bibinfo{year}{2018}\natexlab{}.
\newblock \showarticletitle{{Personalized Recommender Systems for Product-Line
  Configuration Processes}}.
\newblock \bibinfo{journal}{\emph{Computer Languages, Systems \& Structures}}
  \bibinfo{volume}{54} (\bibinfo{year}{2018}), \bibinfo{pages}{451--471}.
\newblock


\bibitem[\protect\citeauthoryear{Polat-Erdeniz, Felfernig, Samer, and
  Atas}{Polat-Erdeniz et~al\mbox{.}}{2019}]%
        {PolatErdeniz2019}
\bibfield{author}{\bibinfo{person}{S. Polat-Erdeniz}, \bibinfo{person}{A.
  Felfernig}, \bibinfo{person}{R. Samer}, {and} \bibinfo{person}{M. Atas}.}
  \bibinfo{year}{2019}\natexlab{}.
\newblock \showarticletitle{{Matrix Factorization based Heuristics for
  Constraint-based Recommenders}}. In \bibinfo{booktitle}{\emph{ACM Symposium
  on Applied Computing (ACM SAC)}}. \bibinfo{publisher}{ACM},
  \bibinfo{address}{Limassol, Cyprus}, \bibinfo{pages}{1655--1662}.
\newblock


\bibitem[\protect\citeauthoryear{Reiter}{Reiter}{1987}]%
        {Reiter1987}
\bibfield{author}{\bibinfo{person}{R. Reiter}.}
  \bibinfo{year}{1987}\natexlab{}.
\newblock \showarticletitle{A theory of diagnosis from first principles}.
\newblock \bibinfo{journal}{\emph{Artificial Intelligence}}
  \bibinfo{volume}{32}, \bibinfo{number}{1} (\bibinfo{year}{1987}),
  \bibinfo{pages}{57--95}.
\newblock


\bibitem[\protect\citeauthoryear{Rodas-Silva, Galindo, Garcia-Gutierrez, and
  Benavides}{Rodas-Silva et~al\mbox{.}}{2019}]%
        {RodasSilva2019}
\bibfield{author}{\bibinfo{person}{J. Rodas-Silva}, \bibinfo{person}{J.
  Galindo}, \bibinfo{person}{J. Garcia-Gutierrez}, {and} \bibinfo{person}{D.
  Benavides}.} \bibinfo{year}{2019}\natexlab{}.
\newblock \showarticletitle{{Selection of Software Product Line Implementation
  Components Using Recommender Systems: An Application to Wordpress}}.
\newblock \bibinfo{journal}{\emph{IEEE Access}}  \bibinfo{volume}{7}
  (\bibinfo{year}{2019}), \bibinfo{pages}{69226--69245}.
\newblock


\bibitem[\protect\citeauthoryear{Sayyad, Ingram, Menzies, and Ammar}{Sayyad
  et~al\mbox{.}}{2013}]%
        {Sayyad2013}
\bibfield{author}{\bibinfo{person}{A. Sayyad}, \bibinfo{person}{J. Ingram},
  \bibinfo{person}{T. Menzies}, {and} \bibinfo{person}{H. Ammar}.}
  \bibinfo{year}{2013}\natexlab{}.
\newblock \showarticletitle{{Scalable product line configuration: A straw to
  break the camel's back}}. In \bibinfo{booktitle}{\emph{{28th IEEE/ACM
  International Conference on Automated Software Engineering (ASE)}}}.
  \bibinfo{address}{Silicon Valley, CA, USA}, \bibinfo{pages}{465--474}.
\newblock


\bibitem[\protect\citeauthoryear{She, Ryssel, Andersen, Wasowski, and
  Czarnecki}{She et~al\mbox{.}}{2014}]%
        {She2014}
\bibfield{author}{\bibinfo{person}{S. She}, \bibinfo{person}{U. Ryssel},
  \bibinfo{person}{N. Andersen}, \bibinfo{person}{A. Wasowski}, {and}
  \bibinfo{person}{C. Czarnecki}.} \bibinfo{year}{2014}\natexlab{}.
\newblock \showarticletitle{Efficient synthesis of feature models}.
\newblock \bibinfo{journal}{\emph{Information and Software Technology}}
  \bibinfo{volume}{56}, \bibinfo{number}{9} (\bibinfo{year}{2014}),
  \bibinfo{pages}{1122--1143}.
\newblock


\bibitem[\protect\citeauthoryear{Temple, Acher, Jézéquel, and Barais}{Temple
  et~al\mbox{.}}{2017}]%
        {Temple2017}
\bibfield{author}{\bibinfo{person}{P. Temple}, \bibinfo{person}{M. Acher},
  \bibinfo{person}{J-M. Jézéquel}, {and} \bibinfo{person}{O. Barais}.}
  \bibinfo{year}{2017}\natexlab{}.
\newblock \showarticletitle{{Learning Contextual-Variability Models}}.
\newblock \bibinfo{journal}{\emph{IEEE Software}} \bibinfo{volume}{34},
  \bibinfo{number}{6} (\bibinfo{year}{2017}), \bibinfo{pages}{64--70}.
\newblock


\bibitem[\protect\citeauthoryear{Temple, Galindo, Acher, and Jézéquel}{Temple
  et~al\mbox{.}}{2016}]%
        {Temple2016}
\bibfield{author}{\bibinfo{person}{P. Temple}, \bibinfo{person}{J. Galindo},
  \bibinfo{person}{M. Acher}, {and} \bibinfo{person}{J-M. Jézéquel}.}
  \bibinfo{year}{2016}\natexlab{}.
\newblock \showarticletitle{Using machine learning to infer constraints for
  product lines}. In \bibinfo{booktitle}{\emph{20th International Systems and
  Software Product Line Conference}}. \bibinfo{publisher}{ACM},
  \bibinfo{address}{Beijing, China}, \bibinfo{pages}{209--218}.
\newblock


\bibitem[\protect\citeauthoryear{Tsang}{Tsang}{1993}]%
        {Tsang1993}
\bibfield{author}{\bibinfo{person}{E. Tsang}.} \bibinfo{year}{1993}\natexlab{}.
\newblock \bibinfo{booktitle}{\emph{Foundations of Constraint Satisfaction}}.
\newblock \bibinfo{publisher}{Academic Press}, \bibinfo{address}{London}.
\newblock


\bibitem[\protect\citeauthoryear{Ulz, Schwarz, Felfernig, Haas, Shehadeh,
  Reiterer, and Stettinger}{Ulz et~al\mbox{.}}{2017}]%
        {Ulz2017}
\bibfield{author}{\bibinfo{person}{T. Ulz}, \bibinfo{person}{M. Schwarz},
  \bibinfo{person}{A. Felfernig}, \bibinfo{person}{S. Haas},
  \bibinfo{person}{A. Shehadeh}, \bibinfo{person}{S. Reiterer}, {and}
  \bibinfo{person}{M. Stettinger}.} \bibinfo{year}{2017}\natexlab{}.
\newblock \showarticletitle{{Human Computation for Constraint-based
  Recommenders}}.
\newblock \bibinfo{journal}{\emph{{Journal of Intelligent Information
  Systems}}}  \bibinfo{volume}{49} (\bibinfo{year}{2017}),
  \bibinfo{pages}{37--57}.
\newblock


\bibitem[\protect\citeauthoryear{von Ahn}{von Ahn}{2005}]%
        {vonAhn2005}
\bibfield{author}{\bibinfo{person}{L. von Ahn}.}
  \bibinfo{year}{2005}\natexlab{}.
\newblock \bibinfo{booktitle}{\emph{Human Computation}}.
\newblock \bibinfo{publisher}{Technical Report CM-CS-05-193}.
\newblock


\bibitem[\protect\citeauthoryear{Wang, Cao, and Wang}{Wang
  et~al\mbox{.}}{2019}]%
        {Wang2019ASO}
\bibfield{author}{\bibinfo{person}{S. Wang}, \bibinfo{person}{L. Cao}, {and}
  \bibinfo{person}{Y. Wang}.} \bibinfo{year}{2019}\natexlab{}.
\newblock \showarticletitle{A Survey on Session-based Recommender Systems}.
\newblock \bibinfo{journal}{\emph{arXiv}}  \bibinfo{volume}{abs/1902.04864}
  (\bibinfo{year}{2019}).
\newblock


\end{thebibliography}
