  Understanding network and application performance are essential for
debugging, improving user experience, and performance comparison. Meanwhile,
modern mobile systems are optimized for energy-efficient computation and
communications that may limit the performance of network and applications. In
recent years, several tools have emerged that analyze network performance of
mobile applications in~situ with the help of the VPN service. There is a
limited understanding of how these measurement tools and system optimizations
affect the network and application performance. In this study, we first
demonstrate that mobile systems employ energy-aware system hardware tuning,
which affects application performance and network throughput. We next show that
the VPN-based application performance measurement tools, such as Lumen,
PrivacyGuard, and Video Optimizer, aid in ambiguous network performance
measurements and degrade the application performance. Our findings suggest that
sound application and network performance measurement on Android devices
requires a good understanding of the device, networks, measurement tools, and
applications.
