  GPUs are readily available in cloud computing and personal devices, but their
use for data processing acceleration has been slowed down by their limited
integration with common programming languages such as Python or Java. Moreover,
using GPUs to their full capabilities requires expert knowledge of asynchronous
programming. In this work, we present a novel GPU run time scheduler for
multi-task GPU computations that transparently provides asynchronous execution,
space-sharing, and transfer-computation overlap without requiring in advance
any information about the program dependency structure. We leverage the GrCUDA
polyglot API to integrate our scheduler with multiple high-level languages and
provide a platform for fast prototyping and easy GPU acceleration. We validate
our work on 6 benchmarks created to evaluate task-parallelism and show an
average of 44% speedup against synchronous execution, with no execution time
slowdown compared to hand-optimized host code written using the C++ CUDA Graphs
API.
