  Describing the relationship between the variables in a study domain and
modelling the data generating mechanism is a fundamental problem in many
empirical sciences. Probabilistic graphical models are one common approach to
tackle the problem. Learning the graphical structure for such models is
computationally challenging and a fervent area of current research with a
plethora of algorithms being developed. To facilitate the benchmarking of
different methods, we present a novel Snakemake workflow, called Benchpress for
producing scalable, reproducible, and platform-independent benchmarks of
structure learning algorithms for probabilistic graphical models. Benchpress is
interfaced via a simple JSON-file, which makes it accessible for all users,
while the code is designed in a fully modular fashion to enable researchers to
contribute additional methodologies. Benchpress currently provides an interface
to a large number of state-of-the-art algorithms from libraries such as
BDgraph, BiDAG, bnlearn, causal-learn, gCastle, GOBNILP, pcalg, r.blip,
scikit-learn, TETRAD, and trilearn as well as a variety of methods for data
generating models and performance evaluation. Alongside user-defined models and
randomly generated datasets, the workflow also includes a number of standard
datasets and graphical models from the literature, which may be included in a
benchmarking study. We demonstrate the applicability of this workflow for
learning Bayesian networks in five typical data scenarios. The source code and
documentation is publicly available from http://benchpressdocs.readthedocs.io.
