  Scientific workflow management systems (SWMSs) and resource managers together
ensure that tasks are scheduled on provisioned resources so that all
dependencies are obeyed, and some optimization goal, such as makespan
minimization, is achieved. In practice, however, there is no clear separation
of scheduling responsibilities between an SWMS and a resource manager because
there exists no agreed-upon separation of concerns between their different
components. This has two consequences. First, the lack of a standardized API to
exchange scheduling information between SWMSs and resource managers hinders
portability. It incurs costly adaptations when a component should be replaced
by a different one (e.g., an SWMS with another SWMS on the same resource
manager). Second, due to overlapping functionalities, current installations
often actually have two schedulers, both making partial scheduling decisions
under incomplete information, leading to suboptimal workflow scheduling.
  In this paper, we propose a simple REST interface between SWMSs and resource
managers, which allows any SWMS to pass dynamic workflow information to a
resource manager, enabling maximally informed scheduling decisions. We provide
an implementation of this API as an example, using Nextflow as an SWMS and
Kubernetes as a resource manager. Our experiments with nine real-world
workflows show that this strategy reduces makespan by up to 25.1% and 10.8% on
average compared to the standard Nextflow/Kubernetes configuration.
Furthermore, a more widespread implementation of this API would enable leaner
code bases, a simpler exchange of components of workflow systems, and a unified
place to implement new scheduling algorithms.
