Skip to main content

Architecture

Overview​

 HTTP client / CLI / SDK
β”‚
β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ rc (host: macOS or Linux) β”‚
β”‚ β”œβ”€ domain: validates job/specβ”‚
β”‚ β”œβ”€ pool: ready VMs per β”‚
β”‚ β”‚ language (jobs only) β”‚
β”‚ β”œβ”€ machines: registry, disk β”‚
β”‚ └─ launcher: spawns the vmm β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚ spawn (never fork)
β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ rc vmm --spec (1 per VM) β”‚
β”‚ libkrun (purego) + β”‚
β”‚ libkrunfw (Linux kernel) β”‚
β”‚ HVF on macOS / KVM on Linux β”‚
β”‚ network gateway (machines β”‚
β”‚ with --net), screen viewer β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚ Unix socket ⇄ vsock port 5000
β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ microVM β”‚
β”‚ job: read-only ext4 root, β”‚
β”‚ /tmp tmpfs, no network β”‚
β”‚ machine: image overlay + β”‚
β”‚ own disk (/dev/vdb) β”‚
β”‚ /opt/runner/guest (static) β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Binaries​

BinaryWhere it runsWhat it does
rchostrun, shell, machine ..., pack ..., serve (API + pool), image build/prune, languages, doctor and the internal vmm subcommand
runner-guestinside the VMlistens on vsock port 5000 and serves each operation: running code, exec, file copy, PTY, workload, published port, secrets, SSH agent, S3 mount. Stored as bin/rc-guest-linux-<arch>

There are two binaries because purego forces dynamic linking against glibc on Linux, and the language images are Alpine (musl). A dynamic guest fails inside the VM with not found; the guest is compiled statically, and scripts/build.sh fails if it comes out dynamic.

Host layers​

The host code follows a hexagonal architecture; cmd/rc only wires the pieces together.

LayerPackageResponsibility
domaininternal/domainjob, result, limits (stdin, files, size, timeout) and E00x errors
applicationinternal/appjob executor and pool; machine lifecycle, branch, staged volumes, credentials, secrets
portsinternal/portsinterfaces (VMSpec and its validation, launcher, catalog, images)
adaptersinternal/adapters/krunlibkrun loading, VM configuration, launcher, vmm process and confinement on Linux
internal/adapters/ociOCI image pull and ext4 build in the build VM
internal/adapters/catalogembedded languages.yaml
internal/adapters/guestprotovsock protocol client
internal/adapters/httpapiHTTP API (jobs and machines)
internal/adapters/machinestoreeach machine's registry and disk (a clone of the base disk)
internal/adapters/runfilereading the runner.code file (TOML)
internal/adapters/netgwuserspace network gateway: egress policy, DNS, ports, ICMP, named networks, credential interception
internal/adapters/screen, viewerthe VM's screen in the browser
internal/adapters/hostfshost side of volumes (file modes for :staged)
othersinternal/credca, internal/objectstore, internal/s3fs, internal/packper-machine CA, S3 client, core of the s3:// volume, pack format
guestinternal/guestvsock server and per-language executor

SDKs: pkg/client (Go) and sdk/python (Python), standard library only.

Path of a job through the API​

  1. POST /api/v1/run reaches httpapi, which builds the job.
  2. The domain validates it: language in the catalog (otherwise E006), stdin ≀ 1,000,000 bytes (otherwise E005), up to 32 files, total code ≀ 4 MiB, valid file names, timeout between 0 and 300 s. A rejection becomes HTTP 400.
  3. The pool hands over an already-booted VM for that language. If the pool is empty, it boots a VM on the spot (cold boot): it never refuses.
  4. The host connects to the Unix socket bound to the VM's vsock port 5000 and sends the job as a frame (4-byte length + JSON).
  5. The guest writes the files to a directory under /tmp, compiles if the language requires it, runs with the job's stdin and timeout, and replies with another frame.
  6. The VM is destroyed (SIGKILL on the process group) and the pool replenishes a new VM in the background.
  7. The API returns status, stdout, stderr, exception, executionTime.

Path of a machine​

  1. rc machine create writes the registry to <cache>/machines/<nome>/machine.json and an ext4 disk cloned from the base disk (clonefile on APFS, FICLONE on btrfs/XFS, a sparse copy everywhere else).
  2. rc machine start launches rc vmm detached (its own session, with no tie to the parent): the machine stays up after the command exits. The pid lives in vmm.pid.
  3. The VM's first process is /opt/runner/machine.sh: it mounts the disk at /persist, an overlayfs with the image underneath and upper/ on top, binds /workspace and runs the guest in a chroot.
  4. exec, shell, cp, logs talk to the guest through the same port 5000 socket. With --net, the VM gets eth0 and traffic goes through the gateway in the vmm process itself, which applies the egress policy.
  5. rc machine stop asks the guest to shutdown; the guest runs sync and exits; if it does not finish within --timeout, SIGKILL.

Details in machine and Network and volumes.

VM lifecycle​

  • The launcher creates the rc vmm --spec <file> process by spawn. A fork of a multi-threaded process is killed by Hypervisor.framework, which is why fork is never used.
  • The vmm process loads libkrun, configures the VM from the spec and calls krun_start_enter, which takes over the process until the VM ends.
  • Job timeout: SIGKILL on the process group.
  • A job VM dies with its parent: Pdeathsig on Linux; on macOS, a watch pipe on fd 3. A machine VM is detached.
  • If the vmm dies during boot, start ends immediately, quoting the last line of vmm.log written during this start.

Details in libkrun.

Images​

Each stack has a read-only ext4 generated from the official OCI image, with the guest, start.sh and the warmup baked in. The build runs in a dedicated Alpine VM. See Images.

Configuration through the environment​

VariableDefaultPurpose
RUNNER_LISTEN127.0.0.1:8080API address (serve)
RUNNER_CACHE_DIR~/Library/Caches/runner-codes (macOS), ~/.cache/runner-codes (Linux)images, build VM, machines, packs and sockets
RUNNER_KRUN_LIB~/.nix-profile/lib/libkrun.dylib (macOS); on Linux ~/.nix-profile/lib/libkrun.so.1 or the libkrun.so.1 sonamelibkrun path
RUNNER_GUEST_BINrc-guest-linux-<arch> next to the host binaryguest injected into the images
RUNNER_SANDBOXonoff disables Landlock + seccomp on Linux, for diagnostics only