  Stacking (or stacked generalization) is an ensemble learning method with one
main distinctiveness from the rest: even though several base models are trained
on the original data set, their predictions are further used as input data for
one or more metamodels arranged in at least one extra layer. Composing a stack
of models can produce high-performance outcomes, but it usually involves a
trial-and-error process. Therefore, our previously developed visual analytics
system, StackGenVis, was mainly designed to assist users in choosing a set of
top-performing and diverse models by measuring their predictive performance.
However, it only employs a single logistic regression metamodel. In this paper,
we investigate the impact of alternative metamodels on the performance of
stacking ensembles using a novel visualization tool, called MetaStackVis. Our
interactive tool helps users to visually explore different singular and pairs
of metamodels according to their predictive probabilities and multiple
validation metrics, as well as their ability to predict specific problematic
data instances. MetaStackVis was evaluated with a usage scenario based on a
medical data set and via expert interviews.
