Longhorn vs OpenEBS for k3s Storage

You’ve got three nodes, empty storage, and a decision to make

Longhorn wins for a straightforward 3-node k3s cluster where backups matter, and OpenEBS wins the moment you want to mix local, replicated, and legacy storage engines per workload.

So you’ve finally got k3s running on that home lab cluster. It’s humming along, pods are deploying, and life is good. Then you realize: stateful workloads need persistent storage, and k3s doesn’t ship with one out of the box. Time to pick between Longhorn and OpenEBS.

This is not a “one-size-fits-all” story. Both are solid choices, but they solve the problem from different angles. Longhorn is the Swiss Army knife that does 80% of what you need with minimal fuss. OpenEBS is the toolbox where you can pick exactly which tool you want for each job. Neither is obviously wrong; it depends on what you’re running, how much you want to tinker, and whether you sleep better knowing your replicas are spread across three nodes or distributed through a local storage + remote replication strategy.

Let’s dig in.

Longhorn: Simple, opinionated, just works

Longhorn is Rancher’s answer to “what if we made distributed block storage boring?” It’s a single operator that manages everything: replicas, snapshots, backups, failover. You install it, point it at your nodes, and it handles the rest.

Think of Longhorn like buying a car with all the features pre-tuned. You get a good experience out of the box because someone already made all the decisions for you.

How Longhorn works

Longhorn runs an agent on each node. When you claim a PersistentVolume, Longhorn launches a controller pod (usually on one node) and replica pods (spread across your cluster). The replicas sync continuously; if a node dies, the controller picks a new replica and keeps serving traffic.

Each volume gets its own controller. Each replica is a container. For a 3-node cluster with five 10GB volumes, you’re spinning up five controllers and fifteen replicas. That’s a lot of little processes, but they’re lightweight.

Installing Longhorn on k3s

# Add the Longhorn Helm repo
helm repo add longhorn https://charts.longhorn.io
helm repo update

# Install with k3s-friendly defaults
helm install longhorn longhorn/longhorn \
  --namespace longhorn-system \
  --create-namespace \
  --set defaultSettings.defaultReplicaCount=2 \
  --set defaultSettings.storageMinimalAvailablePercentage=10 \
  --set defaultSettings.storageOverProvisioningPercentage=200

The key settings:

• defaultReplicaCount=2: For a 3-node cluster, 2 replicas is sane. (3 would eat half your disk space.)
• storageMinimalAvailablePercentage=10: Longhorn won’t accept writes if free space drops below this.
• storageOverProvisioningPercentage=200: Longhorn lets you claim 2x your actual disk space. Reasonable for home lab; adjust down if you’re paranoid.

StorageClass example

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: longhorn
provisioner: driver.longhorn.io
allowVolumeExpansion: true
parameters:
  numberOfReplicas: "2"
  staleReplicaTimeout: "2880"  # 2 days
  fromBackup: ""
volumeBindingMode: WaitForFirstConsumer

Set WaitForFirstConsumer so replicas land on nodes where the pod actually runs. No orphaned replicas in corners of your cluster.

What Longhorn does well

Snapshots & backups: Built-in. Take a snapshot with a kubectl command, back it up to S3 or NFS, restore it anywhere. Your morning self-hosted Nextcloud backup story is trivial.

Failover: Automatic. Node dies, Longhorn’s controller switches to a healthy replica. Your stateful pod reschedules, finds new storage, keeps going.

UI: Longhorn has a dashboard (port 80 on the manager service). You can see your volumes, snapshots, node resources, replica placement. Not fancy, but it works.

Simplicity: One CRD type: Volume. One operator. No plugins to choose, no modes to understand.

The catch

Longhorn’s replica model costs disk space. A 3-replica volume eats 3x the data across your cluster. For a 3-node lab with 500GB per node, that’s only ~500GB of usable storage if you want 3 replicas. Not fun.

Also, Longhorn is controller-heavy. Each volume = one controller pod. If you’ve got thirty small volumes (database, cache, config, logs), you’ve got thirty controllers sitting around. It works, but it’s not elegant.

One more: rebuilding a replica after a node comes back online can saturate your network. If you lose a node with three replica copies and it rejoins, Longhorn starts syncing all of them at once. Your cluster feels slow for a few hours.

OpenEBS: Modular, flexible, more levers to pull

OpenEBS is a storage framework. It doesn’t come with one opinion; it comes with several, and you pick which engine fits each volume. Want local snapshots? Use LocalPV. Want distributed storage? Use cStor or Mayastor. Want a Kubernetes-native NAS? Use OpenEBS NFS.

This is like buying a car and then choosing your own suspension, transmission, and engine separately. Powerful if you know what you’re doing; overwhelming if you just want to drive.

The OpenEBS engines

LocalPV StatefulSet / LocalPV HostPath: Volumes on the local node’s disk. No replication, no distribution. Fast as hell, but if the node dies, so does your data. Good for temporary storage or dev databases where you’re cool with that trade-off.

cStor (legacy, archived): A distributed block storage engine. Data is replicated across pools on different nodes. Similar to Longhorn’s approach, but with more knobs. As of OpenEBS 4.0 it’s been moved to legacy status, it still works and lives on in the openebs-archive org, but it’s no longer part of the main OpenEBS Helm chart. The project pushed everyone toward Mayastor instead.

Mayastor (Replicated PV): OpenEBS’s flagship replicated engine these days. A modern, async Rust engine designed for fast storage. Lower latency than cStor, and at this point (v4.3+) it’s the default, actively maintained path for replicated storage rather than the experimental new kid it used to be. Think: Linux Btrfs vs ZFS, different philosophies.

NFS: OpenEBS can provision NFS shares backed by PVs on your nodes. Useful if you have legacy apps that only speak NFS.

For a 3-node k3s cluster, LocalPV HostPath (dev/temp) or Mayastor / Replicated PV (stateful workloads) are the typical picks today. cStor is still an option if you’re maintaining an older deployment, but for anything new you’d reach for Mayastor.

Installing OpenEBS on k3s

# Add OpenEBS Helm repo
helm repo add openebs https://openebs.github.io/charts
helm repo update

# Install OpenEBS (local engines + replicated Mayastor)
helm install openebs openebs/openebs \
  --namespace openebs \
  --create-namespace \
  --set engines.local.enabled=true \
  --set engines.replicated.mayastor.enabled=true

This enables the LocalPV engines and replicated Mayastor. If you only want LocalPV, leave Mayastor off, fewer components, less overhead. Note that the modern OpenEBS chart (4.x) ships LocalPV and Mayastor; cStor was archived and is no longer installed by this chart, so the examples below that use cStor assume a legacy install from openebs-archive.

LocalPV HostPath StorageClass

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-hostpath
provisioner: local.csi.openebs.io/hostpath
allowVolumeExpansion: false  # hostpath doesn't support expansion
volumeBindingMode: WaitForFirstConsumer

Data lives on whatever node the pod lands on. Expansion not supported (you’d have to manually resize the underlying filesystem). Good for caches, temp data, or apps that can tolerate loss.

cStor StorageClass

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: cstor-single-replica
provisioner: cstor.csi.openebs.io
allowVolumeExpansion: true
parameters:
  cstorPoolCluster: cstor-pool
  replicaCount: "1"
volumeBindingMode: Immediate

For cStor, you first need to provision a CStorPoolCluster:

apiVersion: cstor.openebs.io/v1
kind: CStorPoolCluster
metadata:
  name: cstor-pool
  namespace: openebs
spec:
  pools:
    - nodeSelector:
        kubernetes.io/hostname: node1
      dataRaidGroups:
        - blockDevices:
            - blockDeviceName: /dev/sdb
    - nodeSelector:
        kubernetes.io/hostname: node2
      dataRaidGroups:
        - blockDevices:
            - blockDeviceName: /dev/sdb
    - nodeSelector:
        kubernetes.io/hostname: node3
      dataRaidGroups:
        - blockDevices:
            - blockDeviceName: /dev/sdb

You’re explicitly telling OpenEBS which disk on each node to use. This is where the flexibility comes in, and also where the config complexity lives. You need to know your disk layout, device names, and be comfortable with manual pool provisioning.

What OpenEBS does well

Flexibility: Want different storage engines for different workloads? LocalPV for ephemeral, cStor for important data, NFS for legacy apps. You can do all three.

No replica tax: LocalPV has zero replication overhead. cStor lets you choose replication factor per volume.

Mayastor performance: If you use Mayastor, you get lower latency than Longhorn or cStor. Useful for database-heavy setups.

Expert-friendly: If you know ZFS/storage terminology, cStor’s pool model will make sense to you.

The catch

OpenEBS is more complex. cStor setup requires you to manage BlockDevices, CStorPools, and StorageClasses. You’re doing storage admin work that Longhorn abstracts away.

Documentation is spottier. cStor has been archived in favor of Mayastor, so you’re swimming against the current if you start there for a new cluster. Mayastor is the maintained path now, but it carries more moving parts than Longhorn’s single operator.

LocalPV is simple but risky. You lose a node, you lose data. For a 3-node cluster where one node is your old laptop, maybe don’t use LocalPV for anything that matters.

No built-in disaster recovery like Longhorn. cStor has snapshots, but backups require integration with external tools (Velero, custom scripts). Longhorn’s backup story is nicer out of the box.

The real difference: Architecture

Longhorn is centralized-ish. Each volume has one controller (per node for HA); that controller orchestrates the replicas. If a node goes down, Longhorn’s management components (living elsewhere in the cluster) notice and pick a new replica. Simple, reliable, somewhat synchronous.

OpenEBS (cStor) is more distributed. Pools are standalone; cStor’s management layer is lighter. If a node goes down, the pool on that node is offline until the node comes back or you manually exclude it. You have more control but also more responsibility.

OpenEBS (LocalPV) is the simplest: just use the node’s disk. Pod dies, reschedule on another node, your old data is stranded on the original node until you clean it up.

Performance snapshot

Rough numbers from home lab benchmarking (YMMV):

LocalPV dominates because it’s the node’s disk. cStor beats Longhorn because it’s written in a lower level and avoids some of Longhorn’s scheduler overhead. But for home lab workloads (databases, wikis, document storage), the difference is academic. You’re bottlenecked by disk I/O and network, not by the storage layer.

So which one?

Pick Longhorn if:

• You want to set it and forget it. Install, create a StorageClass, move on.
• Backups matter (databases, media libraries, configs). Longhorn’s backup story is excellent.
• Your cluster is remote or managed by someone who doesn’t want to deal with pool configuration.
• You’ve got spare disk space. Replicas are cheap if you’ve got headroom.
• You’re running 3 to 5 nodes. Longhorn’s replica model scales fine up to moderate cluster sizes.

Pick OpenEBS if:

• You want to optimize storage differently for different workloads. Mix LocalPV (ephemeral), cStor (stateful), NFS (legacy).
• You’re comfortable with storage admin work (pools, block devices, raid groups).
• You’re exploring Mayastor for low-latency database workloads.
• Your disks are precious and you want explicit control over replication.
• You want to learn how distributed storage actually works. cStor teaches you something; Longhorn hides details.

For a 3-node k3s cluster specifically: Longhorn wins unless you have a specific reason to prefer OpenEBS. Three nodes is Longhorn’s sweet spot, big enough for meaningful replication, small enough that one operator handles everything. You set replicaCount=2, point it at your disks, and you’re done. Your 2 AM self will appreciate not having to debug pool membership or missing BlockDevices.

If you’re running a NAS-style setup (media library, file sharing), neither is ideal, both are designed for block storage. Use NFS on a real NAS, or deploy OpenEBS’s NFS engine if you insist on all-in-one.

The pragmatist’s checklist

Before you choose, ask yourself:

1. What are you storing? Databases love distributed storage. Media libraries are fine with local + snapshots. Kubernetes configs? Ephemeral LocalPV is enough.
2. How much disk space do you have? Replicas cost space. If you’ve got 500GB per node, 2 replicas = ~250GB usable. Hurts.
3. Can you afford to lose a node? If yes, consider LocalPV or replicaCount=1. If no, 2+ replicas mandatory.
4. Do you take backups? Longhorn makes this trivial. OpenEBS requires more plumbing.
5. How much time do you want to spend on storage ops? Longhorn: 1 hour setup, 5 minutes maintenance. OpenEBS: 3 hours setup, 15 minutes maintenance.

You’re not picking the objectively best storage engine. You’re picking the one that fits your cluster size, risk tolerance, and patience level. For most 3-node home labs, that’s Longhorn. For labs where someone really wants to learn distributed storage, that’s cStor. Either way, you’re winning. Both beat “just hope your pod doesn’t crash.”

Related Reading

• ArgoCD for Home Lab GitOps
• Capsule: Multi-Tenancy for Home Lab k3s
• Cilium on k3s: When eBPF Networking Pays
• cri-o vs containerd
• Crossplane vs Terraform for Home Lab