  OpenMP is the de-facto standard for shared memory systems in High-Performance
Computing (HPC). It includes a task-based model that offers a high-level of
abstraction to effectively exploit highly dynamic structured and unstructured
parallelism in an easy and flexible way. Unfortunately, the run-time overheads
introduced to manage tasks are (very) high in most common OpenMP frameworks
(e.g., GCC, LLVM), which defeats the potential benefits of the tasking model,
and makes it suitable for coarse-grained tasks only. This paper presents
taskgraph, a framework that uses a task dependency graph (TDG) to represent a
region of code implemented with OpenMP tasks in order to reduce the run-time
overheads associated with the management of tasks, i.e., contention and
parallel orchestration, including task creation and synchronization. The TDG
avoids the overheads related to the resolution of task dependencies and greatly
reduces those deriving from the accesses to shared resources. Moreover, the
taskgraph framework introduces in OpenMP the record-and-replay execution model
that accelerates the taskgraph region from its second execution. Overall, the
multiple optimizations presented in this paper allow exploiting fine-grained
OpenMP tasks to cope with the trend in current applications pointing to
leverage massive on-node parallelism, fine-grained and dynamic scheduling
paradigms. The framework is implemented on LLVM 15.0. Results show that the
taskgraph implementation outperforms the vanilla OpenMP system in terms of
performance and scalability, for all structured and unstructured parallelism,
and considering coarse and fine grained tasks. Furthermore, the proposed
framework considerably reduces the performance gap between the task and the
thread models of OpenMP.
