  Data copy is a widely-used memory operation in many programs and operating
system services. In conventional computers, data copy is often carried out by
two separate read and write transactions that pass data back and forth between
the DRAM chip and the processor chip. Some prior mechanisms propose to avoid
this unnecessary data movement by using the shared internal bus in the DRAM
chip to directly copy data within the DRAM chip (e.g., between two DRAM banks).
While these methods exhibit superior performance compared to conventional
techniques, data copy across different DRAM banks is still greatly slower than
data copy within the same DRAM bank. Hence, these techniques have limited
benefit for the emerging 3D-stacked memories (e.g., HMC and HBM) that contain
hundreds of DRAM banks across multiple memory controllers. In this paper, we
present Network-on-Memory (NoM), a lightweight inter-bank data communication
scheme that enables direct data copy across both memory banks of a 3D-stacked
memory. NoM adopts a TDM-based circuit-switching design, where circuit setup is
done by the memory controller. Compared to state-of-the-art approaches, NoM
enables both fast data copy between multiple DRAM banks and concurrent data
transfer operations. Our evaluation shows that NoM improves the performance of
data-intensive workloads by 3.8X and 75%, on average, compared to the baseline
conventional 3D-stacked DRAM architecture and state-of-the-art techniques,
respectively.
