  We present HAM (Heterogeneous Active Messages), a C++-only active messaging
solution for heterogeneous distributed systems.Combined with a communication
protocol, HAM can be used as a generic Remote Procedure Call (RPC) mechanism.
It has been used in HAM-Offload to implement a low-overhead offloading
framework for inter- and intra-node offloading between different architectures
including accelerators like the Intel Xeon Phi x100 series and the NEC
SX-Aurora TSUBASA Vector Engine. HAM uses template meta-programming to
implicitly generate active message types and their corresponding handler
functions. Heterogeneity is enabled by providing an efficient address
translation mechanism between the individual handler code addresses of
processes running different binaries on different architectures, as well a
hooks to inject serialisation and deserialisation code on a per-type basis.
Implementing such a solution in modern C++ sheds some light on the shortcomings
and grey areas of the C++ standard when it comes to distributed and
heterogeneous environments.
