  The overwhelming majority of High Performance Computing (HPC) systems and
server infrastructure uses Intel x86 processors. This makes an architectural
analysis of these processors relevant for a wide audience of administrators and
performance engineers. In this paper, we describe the effects of hardware
controlled energy efficiency features for the Intel Skylake-SP processor. Due
to the prolonged micro-architecture cycles, which extend the previous Tick-Tock
scheme by Intel, our findings will also be relevant for succeeding
architectures. The findings of this paper include the following: C-state
latencies increased significantly over the Haswell-EP processor generation. The
mechanism that controls the uncore frequency has a latency of approximately 10
ms and it is not possible to truly fix the uncore frequency to a specific
level. The out-of-order throttling for workloads using 512 bit wide vectors
also occurs at low processor frequencies. Data has a significant impact on
processor power consumption which causes a large error in energy models relying
only on instructions.
