  Recent development in lightweight OS-level virtualization, containers,
provides a potential solution for running HPC applications on the cloud
platform. In this work, we focus on the impact of different layers in a
containerized environment when migrating HPC containers from a dedicated HPC
system to a cloud platform. On three ARM-based platforms, including the latest
Nvidia Grace CPU, we use six representative HPC applications to characterize
the impact of container virtualization, host OS and kernel, and rootless and
privileged container execution. Our results indicate less than 4\% container
overhead in DGEMM, miniMD, and XSBench, but 8\%-10\% overhead in FFT, HPCG, and
Hypre. We also show that changing between the container execution modes results
in negligible performance differences in the six applications.
