  Cloud providers offer a variety of execution platforms in form of bare-metal,
VM, and containers. However, due to the pros and cons of each execution
platform, choosing the appropriate platform for a specific cloud-based
application has become a challenge for solution architects. The possibility to
combine these platforms (e.g. deploying containers within VMs) offers new
capacities that makes the challenge even further complicated. However, there is
a little study in the literature on the pros and cons of deploying different
application types on various execution platforms. In particular, evaluation of
diverse hardware configurations and different CPU provisioning methods, such as
CPU pinning, have not been sufficiently studied in the literature. In this
work, the performance overhead of container, VM, and bare-metal execution
platforms are measured and analyzed for four categories of real-world
applications, namely video processing, parallel processing (MPI), web
processing, and No-SQL, respectively representing CPU intensive, parallel
processing, and two IO intensive processes. Our analyses reveal a set of
interesting and sometimes counterintuitive findings that can be used as best
practices by the solution architects to efficiently deploy cloud-based
applications. Here are some notable mentions: (A) Under specific circumstances,
containers can impose a higher overhead than VMs; (B) Containers on top of VMs
can mitigate the overhead of VMs for certain applications; (C) Containers with
a large number of cores impose a lower overhead than those with a few cores.
