  Speculation is fundamental to achieving high CPU performance, yet it enables
vulnerabilities such as Spectre attacks, which remain a significant challenge
to mitigate without incurring substantial performance overheads. These attacks
typically unfold in three steps: they speculatively access sensitive data
(access), alter the cache state (transmit), and then utilize a cache timing
attack (e.g., Flush+Reload) to extract the secret (receive). Most Spectre
attacks exploit a cache timing side channel during the transmit and receive
steps. Our key observation is that Spectre attacks do not require the transmit
instruction to complete before mis-prediction is detected and mis-speculated
instructions are squashed. Instead, it suffices for the instruction to execute
and dispatch a request to the memory hierarchy. Responses from memory that
arrive after squashing occurs still alter the cache state, including those
related to mis-speculated memory accesses. We therefore propose a novel
mitigation technique, Cancellable Memory Requests (CMR), that cancels
mis-speculated memory requests. Immediately upon squashing, a cancellation is
sent to the cache hierarchy, propagating downstream and preventing any changes
to caches that have not yet received a response. This reduces the likelihood of
cache state changes, thereby reducing the likelihood of Spectre attacks
succeeding. We implement CMR on gem5 and show that it thwarts practical Spectre
attacks, and has near-zero performance overheads. We show that CMR can
completely thwart Spectre attacks in four real-world processors with realistic
system configurations.
