__ _
_______ ___ / /____ _(_)__ ___ ____ _______ ___
/ __/ _ \/ _ \/ __/ _ `/ / _ \/ -_) __// __/ _ \/ _ \
\__/\___/_//_/\__/\_,_/_/_//_/\__/_/ (_)__/ .__/ .__/
/_/ /_/
This is my attempt at implementing a simple Linux container runtime from scratch in a Virtual Machine on MacOS/arm64.
- Containers in a Nutshell
- Bare Minimum Objectives
- Virtual Machine Setup
- Experimenting
- Caveats
- References
A container at the most fundamental level on Linux systems is just another process. A process is an instance of a running program.
Now then, what makes containers different from regular processes? You could say that a container has a much more 'contained' and controlled view of the system's resources. This is typically achieved by what's known as Namespaces.
Aptly named, they control what resources a process can 'see'. The second half of the puzzle are Control Groups.
Where namespaces decided what a process can see, Control Groups (or cgroups) decide how much system resources the said process can use. By controlling what system resources a process can see and use, we create the illusion that allows containers to behave as isolated units on the same host kernel.
To make it easy to stay on track. I'm going to define the bare minimum the program must accomplish in order for us to consider its process a container.
- Spawn a process given a command of the form
./container -fs <filesystem-root> -mem 16 -cpu 4 run <cmd> <args> - Ensure that this process has its own namespace (pid, user, mount)
- Ensure that the root mount point is isolated from the parent process
- Enforce memory and CPU limits using cgroups
The reason we specify a -fs or file system flag will become apparent later. The same goes for why we need mount point isolation.
Note: You can skip this step if you're running Linux already.
From here on out, I assume that you're on an arm64 processor, specifically an Apple Silicon. Changing these steps for x86 shouldn't be difficult. If you're unsure what cpu architecture your computer is using, run uname -m. I'm going to make another assumption that you have homebrew installed as well.
Lima will help setup a light weight Linux Virtual Machine on your computer.
brew install lima
limactl startNote: On first run, Lima will ask you to download Ubuntu. I've only experimented on Alpine Linux and the following steps will assume you're running an Alpine box.
Alpine Linux is an extremely lightweight Linux distro commonly used in embedded environments (see busybox). I've chosen this distro because it's simple to use. However, because it's so lightweight we're missing a few tools and binaries, namely some C++ build tools.
sudo apk update
sudo apk add g++ procps make musl-dev cmakeA file system controls file organization and access. You can think of this as a very rudimentary 'docker image'. Of course I'm simplifying, but this is where our container's root will be. The reason for this is to further isolate the container from the host. All the container's ever going to know is that these are the files available to it, nothing else.
limactl shell default
sudo mkdir -p /opt/alpine-rootfs
wget https://dl-cdn.alpinelinux.org/alpine/latest-stable/releases/aarch64/alpine-minirootfs-3.24.0-aarch64.tar.gz
sudo tar -xzf alpine-minirootfs-3.24.0-aarch64.tar.gz -C /opt/alpine-rootfsCompile and run the program inside the Linux Virtual Machine.
g++ -std=gnu++17 container.cpp -o container
./container -fs /opt/alpine-rootfs run /bin/shOr use the 'compile.sh' script after you've given it execution permissions.
On your Lima box, running hostname should give you 'lima-default'
# lima
hostname
> lima-defaultInside the container you should get 'host-container'
#container
hostname
> host-containerHere is the result of running ps on the virtual box then container.
Similarly, running top at the virtual box and then container level.
- No proper error handling, killing on failure.
- Messy argument parsing, see cpu flag code.
- https://youtu.be/8fi7uSYlOdc
- https://en.wikipedia.org/wiki/Linux_namespaces
- https://en.wikipedia.org/wiki/Cgroups
- https://en.wikipedia.org/wiki/BusyBox
- https://en.wikipedia.org/wiki/Systemd
- https://lwn.net/Articles/689856
- https://man7.org/linux/man-pages/man2/wait.2.html
- https://man7.org/linux/man-pages/man7/namespaces.7.html
- https://man7.org/linux/man-pages/man7/cgroups.7.html
- https://linux.die.net/man/2/setgroups
- https://linux.die.net/man/3/uid
- http://man7.org/linux/man-pages/man2/clone.2.html
- https://man7.org/linux/man-pages/man3/sysconf.3.html

