Skip to content
Go back

k3s on Pi 5 Cluster: Real or Toy?

By SumGuy 16 min read
k3s on Pi 5 Cluster: Real or Toy?
Contents

You’ve Got Four Pis and a Dream

A Pi 5 cluster wins if you’re running three to five lightweight services and don’t mind babysitting it, and a mini-PC cluster wins the moment you add Postgres or Nextcloud into the mix.

It’s 2026, and someone on your homelab Discord just posted a picture of five Raspberry Pi 5 boards stacked like a production rack. They’ve got k3s running. They’re talking about “distributed storage” and “ha control plane.” Your first thought: “Is this actually a Kubernetes cluster, or just a very expensive toy?”

It’s both, and neither. k3s on Pi 5 works. I’ve built them. They’re snappy for their size, dirt cheap compared to mini-PCs, and they’ll run actual workloads. But they’ll also eat your lunch in ways you won’t see coming until you’re at 2 AM wondering why your Vaultwarden pod keeps OOMing.

This is the real talk: what actually sticks on Pi clusters, what falls apart, and when you should stop Tetris-ing into smaller hardware and just buy the mini-PC.


Is a Raspberry Pi 5 Cluster Real Kubernetes or Just a Toy?

Pi 5 Specs: The Good News and the Catch

Let’s get baseline expectations straight:

SpecPi 5What This Means
CPU2.4 GHz quad-core Cortex-A76 (BCM2712)Four real cores, but each one is a phone core
RAM2, 4, 8 or 16 GB LPDDR4X8 GB is the sane floor for k3s. 16 GB buys real headroom
Storage (onboard)microSDPure liability for k3s, you’ll thrash
Storage (M.2 HAT)PCIe Gen 2 ×1 by default, Gen 3 ×1 opt-inBasically mandatory. See the HAT section for the Gen 3 flag
VideoHEVC hardware decode only, no hardware encoderAny transcode is a software transcode
NetworkGigabit. PoE+ needs a separate HATFine at cluster scale, but the HAT competes for the GPIO stack
Power draw800 mA typical bare-board active (~4W), 27W PSU recommendedAdd 2 to 4W for an NVMe drive under load
ThermalArm cores throttle from 80°C, hard limit 85°CPassive heatsinks throttle under sustained k3s load. Use the Active Cooler

The catch: you get four real cores, but they are phone cores, and there is no hardware video encoder on the board at all. The bigger trap is storage. microSD is not merely slow for k3s, it is disqualifying: etcd fsyncs on every write, and a card that benchmarks fine on sequential reads will stall for hundreds of milliseconds on those. If you skip the M.2 HAT, you’re not running k3s. You’re watching a slideshow.


Workloads That Actually Work

Here’s what you can realistically run on a 3-node k3s Pi cluster:

Lightweight & Stable

WorkloadCPU PressureRAM PressureStorage I/OVerdict
DNS/AdguardIdleVery lightMinimalNo issues ✓
Jellyfin (direct play)5 to 10%500MB per nodeModerateNo issues ✓
Jellyfin (software transcode)100% of all four cores1.2GB per nodeModerate✗ No hardware encoder
Nextcloud (small)20 to 30%2GBHigh⚠ NVMe needed
Paperless OCR90% (batched)1.5GBVery high⚠ Single job
Vaultwarden5 to 10%400MBMinimalNo issues ✓
Prometheus scrape10 to 15%800MB (small)Moderate⚠ Retention limits
Loki (logs only)20%1GBVery high✗ Don’t try

Workloads you should not run:


The NVMe HAT Game-Changer

This deserves its own section because it will make or break your cluster.

Without NVMe HAT:

microSD → k3s datastore (etcd) thrashing
→ Any stateful workload (Postgres, Redis) → 50ms latencies
→ Logs pile up on root, you hit 90% free space panic

With NVMe HAT:

NVMe (Samsung 970 EVO or equivalent)
├─ /var/lib/rancher/k3s → datastore + node state (actual storage)
├─ /var/log → logs don't fill your root
└─ PVC backing (local-path-provisioner) → workload data

The HAT sits on the PCIe FPC connector. Pi 5 runs that link at PCIe Gen 2 ×1 (5 GT/s) out of the box, which caps you around 400 to 450 MB/s in practice. You can opt into Gen 3 speeds (8 GT/s) and roughly double that:

/boot/firmware/config.txt
dtparam=pciex1_gen=3

Raspberry Pi does not certify the board for Gen 3, and the official docs warn the link may be unstable. Test it under load before you trust etcd to it, and drop back to the default if you see PCIe errors in dmesg. Either way it is plenty for k3s. You’re not NAS-ing, you’re keeping etcd happy.

Pick a drive that physically fits. The official Raspberry Pi M.2 HAT+ takes 2230 or 2242 cards only, and the M.2 HAT+ Compact takes 2230 only. The 2280 sticks most people have in a drawer (Samsung 970/990 EVO and friends) will not mount on it. Either buy a 2230/2242 drive, or use a third-party carrier such as the Pimoroni NVMe Base or a Geekworm X1001-class board, which are built around 2280.

Setup, per node. Stop k3s first: moving a live datastore out from under it is how you corrupt etcd.

Terminal window
sudo systemctl stop k3s
# persist the mount, or the symlink points at an empty directory after reboot
echo "/dev/nvme0n1p1 /mnt/nvme ext4 defaults,noatime 0 2" | sudo tee -a /etc/fstab
sudo mkdir -p /mnt/nvme && sudo mount /mnt/nvme
sudo mv /var/lib/rancher /mnt/nvme/rancher
sudo ln -s /mnt/nvme/rancher /var/lib/rancher
sudo systemctl start k3s

Cost? Around £30 to 50 per Pi for a 250GB to 500GB drive, plus £12 to 20 for the HAT.

One planning trap: the official PoE+ HAT and an M.2 HAT both want the GPIO stack and the space above the board. You cannot just bolt both official boards on. If you want PoE and NVMe on the same Pi, buy a combined PoE+NVMe carrier or feed the Pi from a PoE splitter over USB-C.


Cluster Architecture That Doesn’t Fall Over

Node 1 (k3s server) ← control plane + embedded etcd
├─ etcd (quorum member 1 of 3)
├─ API server
└─ Controller manager + Scheduler
Node 2 (k3s server) ← control plane + embedded etcd
├─ etcd (quorum member 2 of 3)
├─ API server
└─ Controller manager + Scheduler
Node 3 (k3s server) ← control plane + embedded etcd
├─ etcd (quorum member 3 of 3)
├─ API server
└─ Controller manager + Scheduler

Why three? For embedded-etcd HA, all three need to be k3s servers (not agents). Etcd quorum lives on the server nodes. You want an odd number: two nodes need both alive to hold quorum, so one failure takes the cluster read-only and you have bought nothing. Three tolerates one failure. Five tolerates two, for the price of two more Pis. (You can still join pure agent nodes on top for more worker capacity. They just don’t vote.)

Storage architecture:

# local-path-provisioner (built into k3s)
StorageClass: local-path
├─ Uses node's local NVMe
├─ No replication (single-node failure = data loss for that PVC)
└─ Fine for: app configs, cache, temp logs
NOT for: databases you care about, backups
# If you need resilient storage:
# Option A: Longhorn (lightweight, ARM-native)
# - 3 replicas across 3 nodes = 1 node can die
# - Costs ~600MB RAM cluster-wide for metadata
# - Every write goes to 3 replicas over the network, so budget for it
# Option B: just use Postgres on node 1, back it to R2/S3
# - Simpler, faster, but couple your data to one node

Real Performance: What You’ll Actually See

I built a three-node cluster (Pi 5, 8GB each, a 500GB 2242 NVMe on a third-party carrier per node, PoE splitters feeding USB-C rather than PoE HATs). Here’s what it feels like:

Pod startup time:

Database queries (SQLite on NVMe):

Local machine: 5ms
Pi cluster query: 12 to 18ms
(Network + Pi I/O, within reason)

Image pulls (first time):

100MB image, gigabit network: ~5 to 8 seconds
(Pi's CPU + etcd contention: not instant)

Sustained workload (Paperless OCR):

One node @ 100% CPU for 45 sec per PDF
(10-page doc, balanced across cluster with Pod limits)
Other nodes: unaffected, 5 to 10% load

Memory pressure (hitting 7GB on an 8GB node):

kubelet crosses its eviction threshold, evicts the lowest-priority pods
→ 10 to 15 sec later, they reschedule elsewhere if another node has room
→ If no node has room, they sit Pending and you notice immediately

Eviction is orderly right up until the whole cluster is tight, which on three 8GB nodes happens sooner than you expect. Set requests and limits on everything, or the scheduler has no idea what it is packing.

You won’t hit anything that breaks unless you’re stupid about scheduling (e.g., three Nextcloud pods on one node, no anti-affinity rules).


When You’re Actually Outgrowing Pi 5

Spot these warning signs:

Sign 1: I/O Walls

Prometheus scrape taking 15+ seconds
Database queries at 50ms+
Logs dropping on the floor (too much volume)

→ You’ve hit the NVMe/CPU bus limit. Adding more nodes doesn’t help (it’s per-node).

Sign 2: RAM Is Tight

kubelet evicting pods daily (not just under stress)
OOMKills on pods with normal settings
Nothing schedulable without shuffling something else off

→ 16GB is the largest Pi 5, and RAM is soldered, so there is exactly one upgrade available and then you are done. (Disable swap while you are here. Raspberry Pi OS enables dphys-swapfile by default, and letting a node swap turns a fast failure into a slow one.)

Sign 3: CPU Is Actually Maxed

One workload (e.g., Paperless) running at 100% for hours
Affecting other pods' latency
Can't add replicas (CPU would go higher)

→ You need bigger cores. Four Cortex-A76s at 2.4 GHz is what you get, and there is no faster Pi 5.

Sign 4: The Cluster Is Babysitting Your App

You're constantly tuning resource limits
Pod affinity rules to keep things apart
Disabling HA because any node is "critical"
Manual restarts when something wedges

→ You’ve outgrown “hobby cluster” into “production headache on toy hardware.”


Pi 5 vs. Mini-PC Inflection Point

When do you actually need to upgrade?

MetricPi 5 ClusterMini-PC UpgradeJump Reason
Total compute budget£250 to 350 (3 nodes, with HATs and drives)£600 to 900 (3 N100-class mini-PCs)10 to 15W vs 4 to 8W per node, but 2 to 3× perf
Node failure impactStateless pods reschedule. Anything on a local-path PVC is stuck until the node returnsSame failure mode, but you can afford Longhorn or a real NAS backing storeRAM and I/O headroom is what makes replicated storage affordable
Max concurrent workloads3 to 5 (light-medium)15 to 20+ (heavier)Real CPU + RAM per node
”I can leave it alone” hours2 to 3 days (needs monitoring)2 to 3 weeksHeadroom = fewer surprises
Storage backingNVMe over one PCIe lane (400 to 450MB/s, ~900 on Gen 3)NVMe over four lanes (2GB+/s)Four times the lanes
Cost per added node£45 (Pi + NVMe)£200+ (mini-PC + storage)Smaller relative cost increases

Honest inflection point: When you’re running more than one “real” app (Nextcloud + Postgres + Paperless, not just Adguard + Wiki), or you’re tired of monitoring it.

What to buy instead: three N100-class mini-PCs (Beelink, Minisforum and friends: Intel’s own NUC line never shipped an N100), 16GB RAM, 500GB NVMe each, roughly £200 to 250 per box. Same desk footprint, 3 to 4× the performance, x86 so every container image just works. Used enterprise micro desktops (Lenovo ThinkCentre M75q Gen 2 and similar) are cheaper still and take more RAM.


The Real Verdict

k3s on Pi 5 is NOT a toy. It’s a legitimate platform for home lab clusters, and I’ll run it again. You get:

It IS a constrained platform. You’ll hit walls that you can’t architect around:

Pick a Pi cluster if:

Buy a mini-PC instead if:

The Pi 5 isn’t holding you back from real Kubernetes. It’s holding you back from scale. That’s not nothing, but it’s also not a reason to skip it if the workload fits.

Your 2 AM self will decide which one was right.

Common Questions

How much RAM do I need on each Pi 5 for a k3s cluster?

8GB is the practical floor and 16GB is the comfortable answer. k3s itself plus the embedded etcd sits around 700MB to 1GB per server node before you schedule anything. On 8GB you will spend real time tuning requests and limits. RAM is soldered, so buy the size you want up front.

Can I run a k3s Pi 5 cluster from microSD instead of NVMe?

No, not for a cluster you intend to leave running. k3s embedded etcd fsyncs on every write, and microSD handles small synchronous writes badly enough that etcd starts logging slow-apply warnings and losing leader elections. Use the M.2 HAT and an NVMe drive, or run a single-node k3s with the SQLite datastore instead.

Will Jellyfin transcoding work on a Raspberry Pi 5?

No. The Pi 5 has HEVC hardware decode and no hardware video encoder, so Jellyfin falls back to software encoding across all four cores. One 1080p transcode will saturate the node. Run Jellyfin on the Pi as a library server for clients that direct-play, and keep transcoding off the cluster.

Does the Raspberry Pi M.2 HAT+ fit a standard 2280 NVMe drive?

No. The official M.2 HAT+ takes 2230 and 2242 cards, and the M.2 HAT+ Compact takes 2230 only. The common 2280 sticks need a third-party carrier such as the Pimoroni NVMe Base or a Geekworm X1001-class board. Check the form factor before ordering, not after.

Can I use PoE and NVMe on the same Pi 5?

Not with two official HATs. The PoE+ HAT and the M.2 HAT+ both occupy the GPIO stack and the space above the board. Use a combined PoE-plus-NVMe carrier board, or run a PoE splitter that outputs USB-C and power the Pi conventionally while the M.2 HAT keeps the PCIe connector.


Share this post on:

Send a Webmention

Written about this post on your own site? Send a webmention and it'll show up above once verified.


Previous Post
External Secrets Operator + Vault on k3s
Next Post
Kustomize Without Helm: When Overlays Win

Discussion

Powered by Garrul . Sign in with GitHub or Google, or post anonymously.

Related Posts