TrueNAS Finally Stopped Pretending Kubernetes Was Easy
If you’ve been running TrueNAS Scale since the early days, you remember the pain. iX Systems spent years bolting K3s (Kubernetes) onto a storage appliance and calling it a feature. Your storage cluster became a Kubernetes cluster. Your ZFS pools became Kubernetes volumes. Everything was suffocating under YAML, custom resource definitions (CRDs), and the kind of networking complexity that makes you question your life choices at 2 AM.
Late 2024 changed that. TrueNAS Scale pivoted hard: rip out Kubernetes, go native Docker, lean into what makes TrueNAS actually special—ZFS snapshots, dataset granularity, storage-aware app lifecycle. The Apps catalog still exists, but now it’s backed by Docker Compose, not K8s operators. And you can finally install custom apps without learning Helm.
This shift matters because it changes the entire mental model. You’re not running a Kubernetes cluster that happens to have storage. You’re running a storage appliance that can containerize workloads and integrate them with your datasets. Two very different things.
The K3s Era (and Why It Sucked)
For context: TrueNAS Scale shipped with K3s in 2021. The idea made sense on paper—lightweight Kubernetes, container orchestration, enterprise credibility. In practice, it was a forklift to move a couch.
K3s introduced:
- iX-charts: A custom Helm repo for “TrueNAS-optimized” apps. They weren’t. They were Helm charts with hardcoded assumptions and zero flexibility.
- Limited storage integration: ZFS datasets could be mounted as Kubernetes persistent volumes, but the mapping was opaque and divorced from your actual storage strategy.
- No custom apps: If you wanted to run something not in the catalog—your Compose stack, your weird database, your one-off service—you had to SSH in and run Docker directly (which undermined the whole point).
- Upgrade chaos: K3s version bumps broke apps. Apps broke K3s. The entire system felt fragile.
By 2024, even iX Systems had to admit it: K3s wasn’t the answer for a storage appliance. People wanted simplicity, ZFS integration, and the ability to run whatever Docker image they threw at it.
The Docker Pivot (Electric Eel / Fangtooth)
Starting with Electric Eel (24.10, late 2024) and solidified in Fangtooth (25.04), TrueNAS Scale ripped out K3s and went all-in on Docker. This is a major architectural shift, and honestly, it’s the right call.
Now the story is cleaner:
- TrueNAS Apps are curated Docker Compose stacks, packaged as catalogs.
- You can still SSH in and run raw Docker/Compose—no hand-wringing about “am I breaking the TrueNAS cluster?”
- Datasets can be mounted directly into container volumes with first-class ZFS snapshot integration.
- Custom apps are just Compose files. No Helm. No CRDs. No operators.
The apps UI still exists, still looks pretty, still gives you one-click installs. But underneath, it’s Docker. If you need to tweak something, you SSH in, edit the Compose file, and docker compose up. The boundary between “TrueNAS Apps” and “my Docker Compose” has become delightfully blurry.
TrueNAS Apps: What They Are Now
TrueNAS Apps today is a UI layer on top of Docker Compose. When you click “Install Nextcloud” in the web console, here’s what happens under the hood:
- A Compose file is generated from the app definition (or fetched from a catalog).
- Datasets are created to match the app’s storage needs.
docker compose upruns.- The UI watches the container and reports status.
Benefits of using the Apps UI:
- One-click installs: Pick an app, click “Install”, configure a few fields (admin user, storage path, port), done.
- ZFS snapshot integration: Apps tied to specific datasets. The UI can snapshot the dataset, restore it, migrate it.
- Catalog browsing: Curated, tested stacks (Nextcloud, Jellyfin, Immich, Qbittorrent, etc.). Someone else did the Compose work.
- App updates through the UI: Click “Update”, choose a new image tag, watch the container rebuild.
- Easy uninstall: Removes the app, optionally the dataset.
This is genuinely useful for home labs. You’re not hunting down scattered Docker images on GitHub or figuring out which flags -e ADMIN_USER= actually controls.
But here’s the thing: it’s still just Docker underneath. You can SSH in, run docker ps, edit /var/lib/docker/volumes/*/docker-compose.yml, whatever. The Apps UI is a convenience layer, not a cage.
When to Bail: Raw Docker on TrueNAS
The Apps catalog is great for standard workloads. But reach for raw Docker/Compose when:
1. You need custom networking Apps run in a default bridge network. If you need macvlan (for network isolation, passing containers onto your home network as separate IPs), host network (for services that must listen on the host), or cross-container networking with specific DNS—write Compose.
2. You’re wiring up a multi-container stack Say you’re running Home Assistant with PostgreSQL, an MQTT broker, and a custom data processor. The Apps catalog has Home Assistant and Mosquitto individually, but composing them together with custom environment variables, depends_on, and shared networks—that’s a Compose file.
3. Advanced container labels or runtime options GPUs (we’ll get to that), seccomp profiles, resource limits (CPU shares, memory swaps), privileged mode, device passthrough. These are Docker run flags or Compose overrides. Apps UI doesn’t expose them all.
4. The app isn’t in the catalog Obvious, but: if you want to run Ollama, vLLM, an Ansible playbook runner, a reverse proxy with custom Lua scripting—SSH in and bring your own Compose.
5. You’re testing or iterating
Writing a Compose file, tweaking it, breaking it, rebuilding it—you don’t want the Apps UI getting in your way. Raw Docker lets you docker compose down && docker compose up in seconds.
Datasets vs. Docker Volumes: The ZFS Angle
Here’s where TrueNAS Apps actually shine over stock Docker: ZFS-aware storage.
In raw Docker on Linux, volumes are usually just mount points. On TrueNAS with Apps, you can bind volumes to ZFS datasets, which means:
- Snapshots: Right-click a dataset, snapshot it. Now you can restore an app’s data to that point in time without messing with backups.
- Dataset-level policies: Compression, deduplication, quota enforcement—all inherited by the container.
- Visibility:
zfs listshows datasets tied to apps.du -h /var/lib/docker/...doesn’t.
For a TrueNAS App (through the UI), it looks like:
# TrueNAS Apps generates this for youservices: nextcloud: image: nextcloud:28 volumes: - nextcloud-data:/var/www/html/datavolumes: nextcloud-data: driver: local driver_opts: type: nfs o: "addr=localhost,vers=4,soft,timeo=180,bg,tcp" device: ":/mnt/tank/apps/nextcloud/data"When you do this via raw Compose on TrueNAS, you can be even more explicit:
# Raw Compose on TrueNAS Scaleservices: nextcloud: image: nextcloud:28 volumes: - /mnt/tank/apps/nextcloud/data:/var/www/html/data - /mnt/tank/apps/nextcloud/config:/var/www/html/config environment: MYSQL_HOST: mariadb MYSQL_USER: nextcloud MYSQL_PASSWORD: xxx networks: - nextcloud-net depends_on: - mariadb
mariadb: image: mariadb:11 volumes: - /mnt/tank/apps/nextcloud/db:/var/lib/mysql environment: MYSQL_ROOT_PASSWORD: xxx MYSQL_DATABASE: nextcloud networks: - nextcloud-net
networks: nextcloud-net: driver: bridgeThe key difference: host paths instead of named volumes. This ties your app data directly to ZFS datasets. You can snapshot /mnt/tank/apps/nextcloud/data independently. You can set ZFS quotas. You can enable compression per-dataset. You’re leveraging TrueNAS’s storage superpower instead of treating it like a generic Docker host.
Best practice: create a dataset per app (or per app-component), use host paths in Compose, and let ZFS do the heavy lifting.
# Set up a dataset for Nextcloud data with compressionzfs create -p -o compression=lz4 tank/apps/nextcloud/datazfs create -p -o compression=off tank/apps/nextcloud/db # or lz4, depending on workloadThen in Compose:
volumes: - /mnt/tank/apps/nextcloud/data:/var/www/html/data - /mnt/tank/apps/nextcloud/db:/var/lib/mysqlAnd take snapshots through the TrueNAS UI or CLI:
zfs snapshot tank/apps/nextcloud/data@before-upgrade# ... upgrade happens ...zfs rollback tank/apps/nextcloud/data@before-upgrade # if it breaksUpgrades, GPU Passthrough, and Other Realities
App upgrades through the UI: New image tag, click “Update”, container gets recreated with the new image, old volume persists. Straightforward. Just ensure the new version is backwards-compatible with your data.
GPU passthrough: This is where raw Docker/Compose becomes essential. The Apps UI doesn’t expose --gpus flags or runtime: nvidia in Compose. You need to:
- Install NVIDIA container runtime on TrueNAS (nvidia-container-toolkit).
- Write your own Compose:
services:ollama:image: ollama/ollamaruntime: nvidiaenvironment:NVIDIA_VISIBLE_DEVICES: allvolumes:- /mnt/tank/apps/ollama/models:/root/.ollama
- SSH in and
docker compose up.
The Apps UI won’t help you here. But that’s fine—GPU workloads are usually advanced enough that you want direct control anyway.
Upgrades with raw Compose: You own the versioning. Update the image: tag in your Compose file, docker compose down && docker compose up -d, done. No UI, no wizards. Downside: you’re responsible for testing compatibility. Upside: you know exactly what changed.
When the Catalog Wins
Here’s the honest truth: for most self-hosted apps, the TrueNAS Apps catalog is the right choice.
Use TrueNAS Apps when:
- The app is in the catalog (Nextcloud, Jellyfin, Immich, Plex, Home Assistant, Sonarr, Radarr, qBittorrent, Prowlarr, etc.).
- You don’t need exotic networking or GPU passthrough.
- You want the UI to manage the lifecycle—updates, snapshots, uninstall.
- Your stack is app-per-container (not a complex multi-container orchestra).
- You want ZFS snapshots of app data without manual
docker inspectarchaeology.
The catalog covers ~95% of self-hosting use cases. Nextcloud with MariaDB? There’s an app for that (or you install them separately and link them). Home Assistant with add-ons? App. Jellyfin? App. Immich? App.
For the remaining 5%—custom stacks, weird infrastructure, experimental services, GPU workloads—SSH in and use raw Docker. You’re not abandoning TrueNAS. You’re just treating it as what it actually is: a storage appliance that can run containers, not a Kubernetes distro.
The Lesson
TrueNAS Scale’s pivot from Kubernetes to Docker is the industry’s latest reminder that over-engineering is seductive but wrong. K3s was impressive. It had operators, CRDs, auto-scaling. It also made simple things hard and hard things impossible.
Docker is simpler. Less powerful. But on a storage appliance, simple is the point. You want to install an app, bind it to your storage, snapshot it, move on. You don’t want to debug YAML templating at 2 AM because a CRD schema validation rule broke your deployment.
TrueNAS Apps handles 95% of use cases with a UI. Raw Docker handles the other 5% without friction. And ZFS datasets sit underneath it all, snapshotting and deduplicating regardless of what container layer you’re using.
If you’ve been avoiding TrueNAS Scale because of the Kubernetes days, now’s the time to revisit. It’s a storage appliance again.