  As HPC system architectures and the applications running on them continue to
evolve, the MPI standard itself must evolve. The trend in current and future
HPC systems toward powerful nodes with multiple CPU cores and multiple GPU
accelerators makes efficient support for hybrid programming critical for
applications to achieve high performance. However, the support for hybrid
programming in the MPI standard has not kept up with recent trends. The MPICH
implementation of MPI provides a platform for implementing and experimenting
with new proposals and extensions to fill this gap and to gain valuable
experience and feedback before the MPI Forum can consider them for
standardization. In this work, we detail six extensions implemented in MPICH to
increase MPI interoperability with other runtimes, with a specific focus on
heterogeneous architectures. First, the extension to MPI generalized requests
lets applications integrate asynchronous tasks into MPI's progress engine.
Second, the iovec extension to datatypes lets applications use MPI datatypes as
a general-purpose data layout API beyond just MPI communications. Third, a new
MPI object, MPIX stream, can be used by applications to identify execution
contexts beyond MPI processes, including threads and GPU streams. MPIX stream
communicators can be created to make existing MPI functions thread-aware and
GPU-aware, thus providing applications with explicit ways to achieve higher
performance. Fourth, MPIX Streams are extended to support the enqueue semantics
for offloading MPI communications onto a GPU stream context. Fifth, thread
communicators allow MPI communicators to be constructed with individual
threads, thus providing a new level of interoperability between MPI and on-node
runtimes such as OpenMP. Lastly, we present an extension to invoke MPI
progress, which lets users spawn progress threads with fine-grained control.
