  DRAM Main memory is a performance bottleneck for many applications due to the
high access latency. In-DRAM caches work to mitigate this latency by augmenting
regular-latency DRAM with small-but-fast regions of DRAM that serve as a cache
for the data held in the regular-latency region of DRAM. While an effective
in-DRAM cache can allow a large fraction of memory requests to be served from a
fast DRAM region, the latency savings are often hindered by inefficient
mechanisms for relocating copies of data into and out of the fast regions.
Existing in-DRAM caches have two sources of inefficiency: (1) the data
relocation granularity is an entire multi-kilobyte row of DRAM; and (2) because
the relocation latency increases with the physical distance between the slow
and fast regions, multiple fast regions are physically interleaved among slow
regions to reduce the relocation latency, resulting in increased hardware area
and manufacturing complexity. We propose a new substrate, FIGARO, that uses
existing shared global buffers among subarrays within a DRAM bank to provide
support for in-DRAM data relocation across subarrays at the granularity of a
single cache block. FIGARO has a distance-independent latency within a DRAM
bank, and avoids complex modifications to DRAM. Using FIGARO, we design a
fine-grained in-DRAM cache called FIGCache. The key idea of FIGCache is to
cache only small, frequently-accessed portions of different DRAM rows in a
designated region of DRAM. By caching only the parts of each row that are
expected to be accessed in the near future, we can pack more of the
frequently-accessed data into FIGCache, and can benefit from additional row
hits in DRAM. Our evaluations show that FIGCache improves the average
performance of a system using DDR4 DRAM by 16.3% and reduces average DRAM
energy consumption by 7.8% for 8-core workloads, over a conventional system
without in-DRAM caching.
