  GPUs are prevalent in modern computing systems at all scales. They consume a
significant fraction of the energy in these systems. However, vendors do not
publish the actual cost of the power/energy overhead of their internal
microarchitecture. In this paper, we accurately measure the energy consumption
of various PTX instructions found in modern NVIDIA GPUs. We provide an
exhaustive comparison of more than 40 instructions for four high-end NVIDIA
GPUs from four different generations (Maxwell, Pascal, Volta, and Turing).
Furthermore, we show the effect of the CUDA compiler optimizations on the
energy consumption of each instruction. We use three different software
techniques to read the GPU on-chip power sensors, which use NVIDIA's NVML API
and provide an in-depth comparison between these techniques. Additionally, we
verified the software measurement techniques against a custom-designed hardware
power measurement. The results show that Volta GPUs have the best energy
efficiency of all the other generations for the different categories of the
instructions. This work should aid in understanding NVIDIA GPUs'
microarchitecture. It should also make energy measurements of any GPU kernel
both efficient and accurate.
