  Automated Machine Learning (AutoML) frameworks regularly use ensembles.
Developers need to compare different ensemble techniques to select appropriate
techniques for an AutoML framework from the many potential techniques. So far,
the comparison of ensemble techniques is often computationally expensive,
because many base models must be trained and evaluated one or multiple times.
Therefore, we present Assembled-OpenML. Assembled-OpenML is a Python tool,
which builds meta-datasets for ensembles using OpenML. A meta-dataset, called
Metatask, consists of the data of an OpenML task, the task's dataset, and
prediction data from model evaluations for the task. We can make the comparison
of ensemble techniques computationally cheaper by using the predictions stored
in a metatask instead of training and evaluating base models. To introduce
Assembled-OpenML, we describe the first version of our tool. Moreover, we
present an example of using Assembled-OpenML to compare a set of ensemble
techniques. For this example comparison, we built a benchmark using
Assembled-OpenML and implemented ensemble techniques expecting predictions
instead of base models as input. In our example comparison, we gathered the
prediction data of $1523$ base models for $31$ datasets. Obtaining the
prediction data for all base models using Assembled-OpenML took ${\sim} 1$ hour
in total. In comparison, obtaining the prediction data by training and
evaluating just one base model on the most computationally expensive dataset
took ${\sim} 37$ minutes.
