The Disk That Kept Filling Itself
It’s 2 AM. Your agent is three hours into a refactor loop, rebuilding myapp:dev for the fortieth time tonight, and the deploy step just failed with no space left on device. You check df -h and the 250GB root filesystem is pegged at 100%. You didn’t download anything or spin up a database cluster. You just let an AI coding agent iterate on a Dockerfile for a few hours.
This happened on a real box: a 250GB root filesystem filling up once or twice a week, no obvious cause, no giant log file, no forgotten video render. One docker system prune -a --volumes -f reclaimed 215.5GB, most of the disk.
The build cache did not do that; it cleans up after itself. The image store does not clean up at all, and an agent that edits code and rebuilds in a loop is exactly the workload that breaks it fastest. Nothing was watching, because nothing watches by default.
If you already know docker system prune, docker builder prune, and reading docker system df, read Cleaning Up Docker Disk Space the Right Way first. This post picks up where that one stops: why an agentic workflow fills the disk on a schedule a human never would, and how to automate the cleanup instead of doing it by hand every time you get paged.
Full example: Clone the working files at github.com/KingPin/sumguy-examples/docker/docker-disk-churn-ai-builds
Why an Agent Fills a Disk Differently Than You Do
Build an image by hand and you build it, test it, maybe rebuild it a couple times over an afternoon, then move on. An AI coding agent edits a file, rebuilds, runs the container, reads the failure, edits again, rebuilds again. Fifty rebuilds in a single evening is a normal night, not an outlier.
The part that catches people off guard: every one of those rebuilds reuses the same tag. docker build -t myapp:dev . doesn’t append to history. It moves the myapp:dev tag onto the new image, and the image that used to own that tag becomes untagged: a dangling image. The layers underneath it don’t vanish. They sit on disk, unreferenced, waiting for something to notice them.
A human orphans a handful of images a day. An agent looping through a fix-test-fix cycle unattended orphans dozens. Every rebuild is a small leak, and a rebuild loop that runs for hours turns that leak into the flood that ate your weekend.
Two Stores, One Garbage Collector
Docker keeps two disk pools, and only one cleans itself.
The build cache is self-managing. BuildKit runs periodic GC with real default policies. For the docker driver, defaultKeepStorage is 20GB, enforced through four ordered rules: drop ephemeral unused cache older than 48 hours once it passes roughly 14% of that budget (or a 512MB floor), drop unused cache older than 60 days, drop unshared cache over the keep-storage limit, then drop anything still over budget regardless of age. Newer BuildKit versions on non-docker drivers use reservedSpace, maxUsedSpace, and minFreeSpace: roughly 10% of disk (capped at 10GB) reserved, 60% (capped at 100GB) as ceiling, 20GB kept free. This cache also holds RUN --mount=type=cache mounts, the ones that make apt, npm, pip, and go installs fast on rebuild, not just scratch build-context files.
The image store has no garbage collector. None. When a tag moves, the old image sits there until a human or a script tells Docker to remove it. There is no daemon setting, no default policy, no equivalent of BuildKit’s keep-storage budget for images.
That split shows up plainly on a real host running this workload:
$ docker system dfTYPE TOTAL ACTIVE SIZE RECLAIMABLEImages 67 16 28.34GB 18.01GB (63%)Containers 16 15 10.9MB 0B (0%)Local Volumes 58 10 21.48GB 9.589GB (44%)Build Cache 4 0 0B 0BLook at that last row: Build Cache, four records, zero bytes. BuildKit’s GC had already kept it tiny. Meanwhile 18GB of images, 63% of the image store, sat there reclaimable because nothing had ever asked for it back. The build cache was never the problem.
What docker system prune -a --volumes Actually Does
A popular myth here needs correcting, because getting it wrong in either direction is expensive: the claim that docker system prune -a --volumes deletes your named volumes, so don’t run it on anything with real data.
The correction: --volumes on system prune only removes anonymous volumes. The flag’s own description in the Docker docs is “Prune anonymous volumes.” Named volumes, the ones you created with docker volume create mydata or referenced as mydata:/var/lib/postgresql/data in a compose file, survive it either way. Reaching named volumes takes a separate command: docker volume prune -a.
So --volumes is safer than its reputation, but not safe outright. Official database images declare a VOLUME instruction in their own Dockerfile, so docker run postgres with no -v flag gets a silent anonymous volume holding real data, with no name you chose. Remove that container later, forget it existed, and the next --volumes sweep sees an unreferenced anonymous volume and deletes it, data and all.
On the host above, 9.589GB across 48 inactive volumes was sitting reclaimable: the blast radius of running --volumes on a timer without checking first. The rule isn’t “volumes are safe” or “volumes will eat your database.” It’s that --volumes on a schedule is safe only if everything you’d miss is a named volume. Running a database container without an explicit named volume? Fix that before you automate anything.
The Second Tradeoff: Prune Kills Your Cache Mounts Too
Even once you trust the volume behavior, a nightly docker system prune -a has a second cost that’s easy to miss until builds get slow: build cache is always eligible for pruning by system prune, including any RUN --mount=type=cache mounts your Dockerfile uses. Wipe those out every night and your agent’s first rebuild each morning re-downloads every dependency from scratch, because the cache mount that made yesterday’s tenth rebuild instant is gone.
You’re trading disk for build speed, and a nightly full prune picks the wrong side of that trade for an agentic workflow. Prune the thing that’s actually the problem (orphaned images) and leave the thing that already manages itself (build cache) alone.
The Approach Worth Automating
Skip the nightly system prune -a --volumes. Reach for targeted, age-filtered commands instead:
docker container prune -f --filter "until=24h" --filter "label!=keep"docker image prune -af --filter "until=168h" --filter "label!=keep"docker network prune -f --filter "until=24h"The -a matters: without it, image prune only removes dangling (untagged) images. With it, Docker also removes tagged images with no container attached, exactly the pile an agent’s rebuild loop leaves behind. The confirmation prompt says it plainly: “all images without at least one container associated to them.”
Each prune subcommand only accepts the filters it documents. docker image prune, docker container prune, and docker network prune accept until and label (including negated label!=<key>). docker volume prune accepts label only, no until, one more reason to leave volume cleanup out of the timed job: you couldn’t age-gate it if you wanted to.
Different filter keys combine as AND, so --filter until=168h --filter "label!=keep" reads as “older than a week AND not labelled keep.” Add --label keep=true to a build and that image survives every automated sweep. Volumes take the label at creation time instead (docker volume create --label keep=true mydata), which matters the day you decide to add a volume step after all.
The until Filter’s Sharp Edge
until filters on the image’s creation timestamp, not the date you pulled it: for a pulled image, that timestamp comes from whoever built it upstream. The official docs show this exactly: alpine:latest at “8 days ago” survives a 240-hour filter, while debian:jessie at “2 months ago” gets deleted, both judged by build date, not arrival date. A locally built image on a live host reports an age of minutes; a freshly pulled python:3.13-slim reports 6 days old right after the pull, because that’s how long ago the vendor built it.
So pairing -a with until=168h can delete a base image you pulled this morning, if the vendor built it more than a week ago. Actively maintained official images get rebuilt often enough to land inside a 7-day window (python:3.13-slim measured 6 days old on the host above, right after a pull), but anything on a slower release cadence is eligible the moment it arrives. The next build re-pulls it: a slow pull, not lost work. The orphaned layers you actually want gone were built locally minutes ago, so the filter still protects them. Read until=168h as “created more than a week ago, by whoever created it,” not “untouched by me for a week.”
Automating It With a systemd Timer
Systemd ships on Arch, CachyOS, Fedora, RHEL, Debian, and Ubuntu, so a timer unit is the portable answer across all of them. Two files, in /etc/systemd/system/:
[Unit]Description=Automated Docker prune for agent build churnAfter=docker.serviceRequires=docker.service
[Service]Type=oneshot
# Untagged and unused images older than 7 days. This is the one that reclaims# the space an agentic rebuild loop leaves behind.ExecStart=/usr/bin/docker image prune -af --filter until=168h --filter label!=keep
# Stopped containers older than a day.ExecStart=/usr/bin/docker container prune -f --filter until=24h --filter label!=keep
# Networks nothing is attached to.ExecStart=/usr/bin/docker network prune -f --filter until=24h
# Deliberately absent:# --volumes / docker volume prune see README, this is how you lose data# docker builder prune BuildKit's own GC already bounds the build cache## No [Install] section. A timer-triggered oneshot is started by its .timer,# not by a target, so it has nothing to install itself into.[Unit]Description=Run the Docker prune daily
[Timer]# "daily" normalizes to *-*-* 00:00:00. Check any expression with:# systemd-analyze calendar dailyOnCalendar=daily
# Run on the next boot if the machine was off when the timer should have fired.Persistent=true
# Calendar timers already default to an accuracy window of 1 minute. This line# only makes that explicit. Widen it (AccuracySec=1h) to let systemd batch the# wakeup with other timers instead.AccuracySec=1m
[Install]WantedBy=timers.targetThree things here look like mistakes but aren’t.
The .service file has no [Install] section. That’s correct: a oneshot service triggered only by a timer doesn’t need enabling on its own. The timer gets enabled, and pulls the service in when it fires.
Multiple ExecStart lines are fine for Type=oneshot; they run in order, no wrapper script needed to sequence three commands.
And this one bites people coming from cron: systemd does not run ExecStart through a shell, so no pipes, no redirection, no globs. Drop any old > /dev/null 2>&1 from a cron line; there’s no systemd equivalent, and you don’t need one, because journald already captures everything the command prints.
Enable and verify it:
$ systemd-analyze verify /etc/systemd/system/docker-prune.service /etc/systemd/system/docker-prune.timer$ sudo systemctl enable --now docker-prune.timer$ systemctl list-timers docker-prune.timer$ journalctl -u docker-prune.servicesystemd-analyze verify catches typos before you wait for a timer to fire. list-timers shows the next scheduled run. journalctl -u docker-prune.service reads the output that used to vanish into /dev/null under cron, so you can see what got reclaimed instead of trusting it silently.
The Cron Fallback
Not every box runs systemd. Minimal containers, embedded distros, and appliance-style images still lean on cron, and it still works fine for this:
0 2 * * * /usr/bin/docker image prune -af --filter "until=168h" >/dev/null 2>&1Three gotchas bite Docker cron jobs. Cron’s PATH is stripped down, so bare docker often resolves to “command not found” even though it works fine in your interactive shell; use the absolute path, /usr/bin/docker, same as systemd above. A non-root crontab entry needs that user in the docker group, or every invocation fails with a permission denied on the socket. Also watch for a literal %; cron treats it as a newline unless escaped with a backslash.
Rootless Docker Is a Different Animal
Rootless Docker runs the daemon as a per-user process, which changes where this cleanup job lives.
A system-wide unit in /etc/systemd/system/ talks to the system daemon, which doesn’t exist in a rootless setup. You need a user unit instead, in ~/.config/systemd/user/, enabled with systemctl --user enable --now docker-prune.timer.
The trap: user units, by default, only run while that user has an active login session. Log out and the timer stops firing until you log back in, defeating the point. The fix is loginctl enable-linger <user>, which keeps that user’s service manager running with no session open. Skip it and you’ll find the disk full again, wondering why the timer never ran.
Podman has the same shape of problem and the same shape of fix. podman system prune is the equivalent command, and the user unit above is already the right pattern for it, because rootless is Podman’s default rather than an opt-in. For how the two runtimes fit different agent workflows, see Where Should Your Coding Agent Run?.
Common Questions
Does docker system prune --volumes delete named volumes?
No. docker system prune --volumes only removes anonymous volumes, ones Docker created automatically without a name you assigned. Named volumes created with docker volume create or referenced by name in a compose file survive it. Only docker volume prune -a reaches named volumes, and that command is separate from system prune entirely.
How often should the Docker prune timer run?
Daily works for most hosts, including ones running an agentic rebuild loop most of the day. Pair a daily docker image prune -af --filter until=168h with a 7-day age filter so nothing built or used in the last week gets touched. If your disk fills faster than that, shorten the filter window before you shorten the interval.
Will pruning slow down my Docker builds?
Targeted image pruning with an age filter does not slow builds, because it never touches the build cache. A full docker system prune -a does slow builds, because it also wipes RUN --mount=type=cache mounts, forcing a fresh dependency download on the next build. Prune images and containers on a timer; leave the build cache to BuildKit’s own garbage collector.
Do I need this cleanup automation on Podman?
Yes. Rebuilding the same tag orphans the previous image under Podman exactly as it does under Docker, because that behavior comes from how tags work rather than from the engine. Use podman system prune with the same age filters, and install it as a user timer rather than a system one, since Podman runs rootless by default.
