  This paper presents the design of AgilePkgC (APC): a new C-state architecture
that improves the energy proportionality of servers that operate at low
utilization while running microservices of user-facing applications. APC
targets the reduction of power when all cores are idle in a shallow C-state,
ready to transition back to service. In particular, APC targets the power of
the resources shared by the cores (e.g., LLC, network-on-chip, IOs, DRAM) which
remain active while no core is active to use them. APC realizes its objective
by using low-overhead hardware to facilitate sub-microsecond entry/exit latency
to a new package C-state and judiciously selecting intermediate power modes for
the different shared resources that offer fast transition and, yet, substantial
power savings. Our experimental evaluation supports that APC holds the
potential to reduce server power by up to 41% with a worst-case performance
degradation of less than 0.1% for several representative workloads. Our results
clearly support the research and development and eventual adoption of new deep
and fast package C-states, like APC, for future server CPUs targeting
datacenters running microservices.
