Using Kata Containers with Docker

Introduction:

Containers aren't actually some kind of magic; they are simply an isolation layer that utilizes the namespaces and cgroups features built right into the Linux kernel. Thanks to this, they isolate the container you are running and the processes executing inside it from the host system itself. Their main difference from VMs (virtual machines) is that instead of spinning up a separate, brand-new system on top of your existing host, they share the host kernel, allowing for much faster startup times and effortless usability.

The Namespace part separates the container above from the running parts of the underlying system. Examples of its main components include:

• PID -> The container still gets its own PID 1 (the root process) and is completely unaware of the processes running on the host system.
• NET -> The container doesn't know about the host system's network, ports, or routing operations.
• MNT -> The container can see and use specific file paths from the host system, but it knows nothing about anything outside of what it's permitted to see.
• IPC -> The container cannot access the host system's shared memory paths or IPC resources outside of its own processes.
• UTS -> The container can have a different hostname than the host system.
• USER -> A user or group inside the container can be mapped to a user or group on the host system with much fewer privileges.

The Cgroups part, on the other hand, is used to put limits on resources like CPU, RAM, etc., for this container.

Why Kata Containers/Runtime:

Standard Docker containers are actually excellent—they are lightweight and can boot (start) incredibly fast. However, they share the underlying host system's kernel (host kernel). If an attacker somehow manages to achieve a container escape, they could exploit vulnerabilities within the kernel to gain access to the host system or other containers. This is exactly where Kata Containers provides a solution. It converts the container into a microVM as we know it, isolating it a step further from the underlying host system. Of course, just like the performance trade-offs that come with any security layer, this approach has its losses in certain areas as well. It utilizes QEMU, cloud-hypervisor, or Firecracker (AWS) to run these microVMs. Below, I will share some brief, straightforward benchmark results so we can see what the main performance trade-offs look like. Running a benchmark inside the container wouldn't make much sense; instead, the things to look at could be the container's boot time and its memory footprint.

There are some other runtimes that can be used for docker, check this link

Benchmark:

Tested runtimes:

• kata (qemu)
• kata (cloud-hypervisor)
• docker default (runc)

Test Method: For each of the runtimes specified above

• An nginx:alpine container will be started, and it will wait just long enough to return the default page via a curl request. This duration will be recorded.
• The mem value returned by docker stats will be captured.
• The RSS value (resident set size) of the hypervisor process will be captured.

Each runtime (cloud-hypervisor, qemu, runc) will run for 2 rounds sequentially, repeating 10 times in each round. In total, the average of 20 measurements will be taken.

Environment:

• Kata Containers Version: 3.32.0 link
• Docker Version: 29.6.1, build 8900f1d
• Host OS: Ubuntu 24.04.4 Linux 6.8.0-134-generic #134-Ubuntu SMP PREEMPT_DYNAMIC
• Host Specifications: 8 vCPU 1 Numa Node, 16GB RAM

Test Results:

The Result:

For a small application, it's impossible not to notice the difference that using Kata and MicroVM containers makes in boot time and memory usage. With boot times increasing by 12–17x and a cost of 145–266MB of memory per container, the overhead is simply too high for an application like Nginx.

In a direct Kata-to-Kata comparison, Cloud-Hypervisor boots 30% faster than QEMU and uses half the memory. For Kata Containers, Cloud-Hypervisor might be the preferred choice. Even so, the resulting load for servers running hundreds of containers is genuinely high. The Kata runtime's Firecracker VMM should also be added to this comparison; perhaps due to its serverless approach, Firecracker could achieve even lower memory consumption there.

In summary, adopting Kata Containers is a step that must be taken with a clear understanding of why we are making that choice. It makes real sense if we truly need VM-level isolation... If no such need exists and it is selected for dense multi-tenant deployments, it will definitely create bottlenecks in other areas. Let's not forget that security is a continuous process. Even with this choice, one could still encounter vulnerabilities regarding KVM escape.

References:

• CNCF Landscape
• Kata Containers
• Docker Runtimes

Thanks: Main Photo: Regine Tholen on Unsplash