  The vast majority of high-performance embedded systems implement multi-level
CPU cache hierarchies. But the exact behavior of these CPU caches has
historically been opaque to system designers. Absent expensive hardware
debuggers, an understanding of cache makeup remains tenuous at best. This
enduring opacity further obscures the complex interplay among applications and
OS-level components, particularly as they compete for the allocation of cache
resources. Notwithstanding the relegation of cache comprehension to proxies
such as static cache analysis, performance counter-based profiling, and cache
hierarchy simulations, the underpinnings of cache structure and evolution
continue to elude software-centric solutions. In this paper, we explore a novel
method of studying cache contents and their evolution via snapshotting. Our
method complements extant approaches for cache profiling to better formulate,
validate, and refine hypotheses on the behavior of modern caches. We leverage
cache introspection interfaces provided by vendors to perform live cache
inspections without the need for external hardware. We present CacheFlow, a
proof-of-concept Linux kernel module which snapshots cache contents on an
NVIDIA Tegra TX1 SoC (system on chip).
