One Bad Plugin Should Not Own Your Git Server Too
Here is the setup on my box: one Linux host, roughly 20 Unix user accounts, and one service group per account. Blogging stack under one user. Git hosting under another. Document management, uptime monitoring, change detection, the usual homelab pile, each in its own account. Every one of those users runs its own rootless Docker daemon and owns every container underneath it. No service stack runs as root. Nothing shares a daemon with anything else.
The verdict up front: this gets you most of the isolation of one VM per service, at a fraction of the cost of actually running 20 VMs, and the residual risk you’re left holding is a kernel-level privilege escalation. On a single-host homelab or a solo-operator box, that trade is worth making. I have run it this way for three years.
Compare that to what most people actually run: one Docker daemon, rootful, every stack living under it. Membership in the docker group is root-equivalent: anyone in that group can mount the host filesystem into a container and read anything on the box. One compromised blog plugin with a container escape does not just own the blog. It owns the git server, the document store, and every secret sitting in every other container’s environment, because they all sit behind the same daemon socket. That is the failure mode worth designing around.
Four Ways to Run Twenty Services on One Box
Option 1: one rootful daemon, everything under it
This is the default. You install Docker once, add yourself to the docker group, and every docker-compose.yml on the box talks to the same daemon as root. It’s the fastest to set up and the worst to compromise. One escape, one group membership, one daemon socket: game over for every container on the host. Most homelabs run this way because nobody tells them the blast radius until it’s too late.
Option 2: one Unix user per service group, rootless Docker each
The one I run. Twenty Unix accounts, twenty rootless daemons, twenty separate UID mappings. A container escape in the git user’s stack lands the attacker as the git user, not as root, and not as any of the other 19 users. File permissions do the rest of the boundary work for free. This wins for the single-host, solo-operator case, which describes most homelabs and a good chunk of small production boxes too.
Option 3: one VM per service group
Real kernel isolation. A KVM guest running the docs stack cannot touch the git guest’s memory or its kernel, because they don’t share one. This wins the moment you have hostile multi-tenant workloads: customers you don’t trust, compliance language that requires hardware-level separation, or workloads where a shared-kernel escape is not an acceptable risk at any probability. It also means 20 kernels to patch, 20 sets of apt upgrade to remember, and 20 times the idle RAM overhead. Most home operators try this once, keep two VMs current, and quietly let the other 18 rot.
Option 4: Kubernetes or k3s
Wrong tool for one box. Kubernetes buys you scheduling across nodes, rolling deploys across a fleet, and API-driven infrastructure. On a single node it buys you an etcd or SQLite state store, a control plane using RAM your services could be using, and none of the multi-node benefits that justify the complexity. If you’re not spreading load across at least two or three physical hosts, skip it. Compose plus 20 Unix users does the same isolation job with none of the YAML tax.
How the Isolation Actually Works
The mechanism doing the real work here is Linux user namespaces, and specifically the /etc/subuid and /etc/subgid files. Every Unix user that runs rootless Docker gets its own range of subordinate UIDs and GIDs, and container UID 0 gets mapped to a different host UID for every user. This is the part most people who copy a rootless Docker tutorial never actually understand, and it’s the part that makes the whole design hold together.
Fedora and RHEL hand out these ranges automatically when useradd runs. Debian and Ubuntu often do not, and that gap is the first thing to bite anyone following a rootless tutorial on those distros. The allocation is 65,536 subordinate UIDs per user (SUB_UID_COUNT in /etc/login.defs), starting at UID 100000 and stepping forward for each new account:
svcblog:100000:65536svcgit:165536:65536Inside svcblog’s containers, process UID 0 (container root) maps to host UID 100000. Inside svcgit’s containers, container UID 0 maps to host UID 165536. Those are two different, unprivileged host users. A process that breaks out of the svcblog container and finds itself running as “root” is actually running as host UID 100000, an account with no sudo, no group memberships worth having, and read access to exactly the files svcblog owns. It cannot read svcgit’s files, cannot touch svcgit’s containers, and cannot see svcgit’s Docker socket, because none of those things are group-readable by UID 100000.
Compare that to a rootful daemon with docker group membership: there is no subuid mapping to escape into, because there was never a boundary in the first place. Root in the container was root on the host the whole time; the container runtime’s namespacing hid that fact from you, not from an attacker who breaks it.
The other three practical wins ride on top of that mechanism:
- Ordinary file permissions become the trust boundary. No AppArmor profile to author, no SELinux policy language to learn.
chmodandchownalready do the job, because every service’s files are owned by a distinct UID. - Backup and teardown match the security boundary.
rsync -a /home/svcgit/ backup/is a complete, self-contained backup of the git stack. Deleting thesvcgituser deletes the stack, its data, and its daemon state in one command. - One kernel to patch, so it actually gets patched. Twenty VMs means twenty separate operating systems to keep current, and by month four most homelab operators have stopped updating 18 of them. This is a human-factors argument, not a technical one, but it’s the argument that actually predicts what happens on a real box.
Setting It Up
Create the service user and confirm its subuid/subgid allocation before touching Docker:
sudo useradd --create-home --shell /bin/bash svcgitgrep ^svcgit: /etc/subuidgrep ^svcgit: /etc/subgid# svcgit:165536:65536 in both files means you're set
# empty output? Debian and Ubuntu usually skip this. Allocate the range yourself:sudo usermod --add-subuids 165536-231071 --add-subgids 165536-231071 svcgitGive the account a real login shell rather than /usr/sbin/nologin. Rootless Docker needs a user session and a systemd user manager, and nologin blocks both, which turns the very next command into a permission error.
Install rootless Docker for that user. The full walkthrough, including the uidmap package prerequisite and the dockerd-rootless-setuptool.sh script, is already covered in Rootless Docker: Run Without Root. Run it as svcgit, and not as yourself:
sudo -u svcgit -idockerd-rootless-setuptool.sh installThat script prints the socket path you’ll need for everything downstream:
export DOCKER_HOST=unix:///run/user/1000/docker.sockDo not stop here. Before you log out, enable lingering for the user, or the daemon dies the moment the session does:
sudo loginctl enable-linger svcgitloginctl show-user svcgit --property=Linger# Linger=yes confirms it stuckA representative compose file for one service group, with paths scoped to that user’s home directory:
services: gitea: image: gitea/gitea:1.24 restart: unless-stopped environment: - USER_UID=1000 - USER_GID=1000 volumes: - /home/svcgit/gitea/data:/data - /home/svcgit/gitea/config:/etc/gitea ports: - "127.0.0.1:3001:3000" - "127.0.0.1:2222:22"Bring it up as the service user, from that user’s own shell, with the rootless daemon already running:
sudo -u svcgit -icd ~/gitea && docker compose up -dDriving Updates Across Twenty Daemons
I use hoist, a label-driven Docker update tool, to keep images current across every user’s stack. I already wrote the full rundown of how hoist’s labels, per-container config, and notification channels work in Hoist: Label-Driven Docker Updates, so I won’t repeat it here. The part that’s specific to this setup is pointing one updater at 20 separate daemons.
The pattern I use is one file per user in /etc/cron.d/, each one targeting that user’s own socket:
10 6 * * * svcgit DOCKER_HOST=unix:///run/user/$(id -u)/docker.sock /usr/local/bin/hoist >> /var/log/cron_logs/svcgit_hoist.log 2>&1Two details matter more than they look like they should. First, use $(id -u) instead of a hardcoded numeric UID. Cron hands the command line to sh, so the substitution resolves at run time, and it keeps resolving correctly even if you ever delete and recreate the user with a different UID. A hardcoded UID in that line just silently points at a socket that no longer exists, and the job fails quietly forever.
Second: redirecting output to a log file means a failed update run is invisible until you go looking for it, which is another way of saying never. Point hoist at a notification channel instead. I use hoist’s Discord webhook support so a failed pull or a broken recreate lands in a channel I actually look at, instead of a log file rotting in /var/log.
The alternative worth knowing about is a systemctl --user timer per user. It inherits that user’s environment automatically, including the rootless Docker CLI context set up during install, so it needs no explicit DOCKER_HOST export at all. What you lose is the one-directory overview: 20 /etc/cron.d files are all greppable from one place as root, where 20 user-scoped systemd timers are scattered across 20 home directories. Neither approach is wrong. I use cron.d because I’d rather grep one directory than SSH into 20 sessions to check a schedule.
Where This Design Leaks
This is the part that gets glossed over in every rootless Docker writeup I’ve read, so I’m giving it the weight it deserves.
One kernel means one privilege escalation breaks all 20 users at once. Rootless Docker requires unprivileged user namespaces to be enabled in the kernel, and that feature has been a recurring source of local privilege escalation bugs. Three worth knowing: CVE-2022-0185, a heap buffer overflow in the kernel’s filesystem context parsing exploitable by an unprivileged user once user namespaces are available. CVE-2023-0386, an OverlayFS UID-mapping flaw that lets an unprivileged user smuggle a setuid binary from a namespaced mount into the host filesystem. CVE-2023-32233, a use-after-free in netfilter’s nf_tables reachable once CONFIG_USER_NS is enabled. All three needed unprivileged user namespaces turned on to be reachable by a non-root local user, which is exactly the kernel feature rootless Docker depends on. The trade you’re making is real: you swap “the docker group is root-equivalent” for “unprivileged user namespaces are enabled,” and that second setting has its own CVE history. The trade still favors the operator over one shared rootful daemon. It is not free, and anyone telling you it is hasn’t read the CVE list.
TCP is not isolated between users. Files and processes get separated by the UID mapping. Listening sockets do not. If svcgit publishes a port on 127.0.0.1:3001, any other local user on the box, including svcblog or svcdocs, can open a connection to it. Nothing about rootless Docker or subuid ranges stops that, and most people who build this setup never notice, because everything still works. Three fixes, pick based on what the service supports:
- Bind each user’s services to a distinct loopback alias (
127.0.0.1for one user,127.0.0.2for the next) instead of sharing127.0.0.1. - Use a Unix socket instead of a TCP port where the app supports it. No listening port means no cross-user reachability question at all.
- Restrict by socket owner with nftables. This rule drops connection attempts to
svcgit’s published port from any process not owned bysvcgitor by root:
table inet svcgit_guard { chain output { type filter hook output priority 0; policy accept; ip daddr 127.0.0.1 tcp dport 3001 meta skuid != { 0, "svcgit" } drop }}Keep UID 0 in that set. Drop it and you also block a rootful reverse proxy from reaching the backend, which breaks every site on the box in a way that looks like a DNS problem for the first twenty minutes.
No shared Docker networks across users. A reverse proxy sitting in svcmon’s daemon cannot join svcgit’s bridge network, because bridge networks are scoped to the daemon that created them. The proxy has to reach every backend through a published loopback port, which is the exact hole the previous point describes. This is a structural cost of running 20 separate daemons. The reverse proxy ends up being the seam where this design’s isolation gets thinnest.
No resource ceilings by default. A runaway OCR job in the docs user’s container can starve the git user for CPU and memory, because nothing caps either one against the other out of the box. Fix it at the systemd slice level, which persists across reboots:
sudo systemctl set-property user-$(id -u svcgit).slice MemoryMax=4G CPUQuota=200%Operational overhead is real. Twenty daemons, twenty socket paths, twenty sets of logs, twenty update runs. This only stays manageable because it’s automated: one cron pattern, one notification channel, one script to bootstrap a new service user. Skip the automation and you will stop maintaining most of these accounts within a few months, the same way people stop patching 18 out of 20 VMs.
The Lingering Trap
loginctl enable-linger <user> is not optional, and forgetting it is the single most common way this whole setup breaks. Without it, systemd tears down /run/user/<uid> the moment that user has no active login session. Two consequences follow, and both are nasty because they’re invisible until something’s already broken:
- Containers do not come back after a reboot. The rootless daemon never restarts, because its user manager never starts, because nobody logged in.
- Any cron job or script pointing
DOCKER_HOSTatunix:///run/user/<uid>/docker.sockfails outright, because the socket path doesn’t exist anymore. It fails silently if the output is redirected to a log file nobody reads, which, per the update section above, is exactly the mistake to avoid twice over.
Check it, don’t assume it:
loginctl show-user svcgit --property=LingerLinger=no means the daemon is one reboot away from not coming back. Fix it once per user, right after creating the account, and you never think about it again.
When You Should Not Do This
Skip this design if you’re facing hostile multi-tenant workloads: customers whose containers you don’t control the contents of, workloads where a shared-kernel escape is a business risk rather than a homelab annoyance, or compliance language that specifically requires hardware-level tenant separation. That’s option 3’s job, one VM per tenant, and no amount of subuid cleverness substitutes for a separate kernel when the threat model actually calls for one.
Skip it too if you’re running across multiple physical hosts, or you expect to soon. Twenty Unix users work fine on one box; they don’t coordinate across two. That’s Kubernetes or Nomad territory, and pretending otherwise just means reinventing a worse version of their scheduler by hand.
For everything in between, one box, one operator, a pile of self-hosted services you actually trust the containers of, this design is the right trade between “one daemon owns everything” and “twenty kernels I’ll stop patching.”
Common Questions
Is rootless Docker slower than rootful Docker?
Yes, for network-heavy workloads. Rootless Docker routes traffic through slirp4netns, a userspace network stack that adds overhead on every packet compared to rootful Docker’s kernel bridge. Disk and CPU-bound container work show no meaningful difference. For typical self-hosted services this never matters. Benchmark first if you run a high-throughput proxy or database.
Can rootless Docker bind to port 80?
No, not by default. Unprivileged processes cannot bind ports below 1024 unless you lower net.ipv4.ip_unprivileged_port_start or grant CAP_NET_BIND_SERVICE to the rootlesskit binary. Most multi-user setups skip both and run one reverse proxy on 80 and 443, forwarding to each service user’s published high port.
How many Unix users is too many for one host?
Idle daemon memory and operator attention set the ceiling, well before any kernel limit does. Each rootless Docker daemon runs dockerd, containerd, and rootlesskit as separate processes, costing roughly 50 to 100MB before a single container starts. Twenty daemons sit near one to two gigabytes. Past 20 or 30 accounts, tracking schedules and backups by hand stops being reliable.
Do I need a separate Unix user for every single container?
No. Group Unix users by trust boundary, not by container count. One user per Compose stack that shares a single purpose gives you the isolation that matters, without 40 accounts to manage for eight real services. Split a group further only when two services inside it should not reach each other’s data.
