  Cache plays a critical role in reducing the performance gap between CPU and
main memory. A modern multi-core CPU generally employs a multi-level hierarchy
of caches, through which the most recently and frequently used data are
maintained in each core's local private caches while all cores share the
last-level cache (LLC). For inclusive caches, clean cache lines replaced in
higher-level caches are not necessarily copied back to lower levels, as the
inclusiveness implies their existences in lower levels. For exclusive and
non-inclusive caches that are widely utilized by Intel, AMD, and ARM today,
either indiscriminately copying back all or none of replaced clean cache lines
to lower levels raises no violation to exclusiveness and non-inclusiveness
definitions.
  We have conducted a quantitative study and found that, copying back all or
none of clean cache lines to lower-level cache of exclusive caches entails
suboptimal performance. The reason is that only a part of cache lines would be
reused and others turn to be dead in a long run. This observation motivates us
to selectively copy back some clean cache lines to LLC in an architecture of
exclusive or non-inclusive caches. We revisit the concept of reuse distance of
cache lines. In a nutshell, a clean cache line with a shorter reuse distance is
copied back to lower-level cache as it is likely to be re-referenced in the
near future, while cache lines with much longer reuse distances would be
discarded or sent to memory if they are dirty. We have implemented and
evaluated our proposal with non-volatile (STT-MRAM) LLC. Experimental results
with gem5 and SPEC CPU 2017 benchmarks show that on average our proposal yields
up to 12.8% higher throughput of IPC (instructions per cycle) than the
least-recently-used (LRU) replacement policy with copying back all clean cache
lines for STT-MRAM LLC.
