  In this paper we present MLaut (Machine Learning AUtomation Toolbox) for the
python data science ecosystem. MLaut automates large-scale evaluation and
benchmarking of machine learning algorithms on a large number of datasets.
MLaut provides a high-level workflow interface to machine algorithm algorithms,
implements a local back-end to a database of dataset collections, trained
algorithms, and experimental results, and provides easy-to-use interfaces to
the scikit-learn and keras modelling libraries. Experiments are easy to set up
with default settings in a few lines of code, while remaining fully
customizable to the level of hyper-parameter tuning, pipeline composition, or
deep learning architecture.
  As a principal test case for MLaut, we conducted a large-scale supervised
classification study in order to benchmark the performance of a number of
machine learning algorithms - to our knowledge also the first larger-scale
study on standard supervised learning data sets to include deep learning
algorithms. While corroborating a number of previous findings in literature, we
found (within the limitations of our study) that deep neural networks do not
perform well on basic supervised learning, i.e., outside the more specialized,
image-, audio-, or text-based tasks.
