  Partitioning applications between NDP and host CPU cores causes inter-segment
data movement overhead, which is caused by moving data generated from one
segment (e.g., instructions, functions) and used in consecutive segments. Prior
works take two approaches to this problem. The first class of works maps
segments to NDP or host cores based on the properties of each segment,
neglecting the inter-segment data movement overhead. The second class of works
partitions applications based on the overall memory bandwidth saving of each
segment, and does not offload each segment to the best-fitting core if they
incur high inter-segment data movement. We show that 1) mapping each segment to
its best-fitting core ideally can provide substantial benefits, and 2) the
inter-segment data movement reduces this benefit significantly.
  To this end, we introduce ALP, a new programmer-transparent technique to
leverage the performance benefits of NDP by alleviating the inter-segment data
movement overhead between host and memory and enabling efficient partitioning
of applications. ALP alleviates the inter-segment data movement overhead by
proactively and accurately transferring the required data between the segments.
This is based on the key observation that the instructions that generate the
inter-segment data stay the same across different executions of a program on
different inputs. ALP uses a compiler pass to identify these instructions and
uses specialized hardware to transfer data between the host and NDP cores at
runtime. ALP efficiently maps application segments to either host or NDP
considering 1) the properties of each segment, 2) the inter-segment data
movement overhead, and 3) whether this overhead can be alleviated in a timely
manner. We evaluate ALP across a wide range of workloads and show on average
54.3% and 45.4% speedup compared to only-host CPU or only-NDP executions,
respectively.
