Prometheus Is Great Until Your RAM Bill Says Otherwise
VictoriaMetrics wins the moment you’re RAM-constrained or need more than 15 days of retention, and Prometheus stays fine if you only need two weeks of data and sub-second queries.
Prometheus is the monitoring darling of the 2020s. Thousands of self-hosted labs and small production clusters run it without breaking a sweat. Battle-tested. Simple. Built-in service discovery for everything under the sun. The tooling ecosystem is thick, Grafana dashboards, alert rules, exporters for your cat’s water bowl.
But Prometheus has a problem, and it’s called memory hunger.
Prometheus stores all active metrics in RAM. All of them. By design. That’s why you get sub-second query latency: the entire TSDB lives hot in memory, ready to serve. A single-node Prometheus with 10 million time series? You’re looking at 15 to 20 GB of RAM, easy. Default retention is 15 days. Disk I/O is minimal. But your retention window? Hard cap. After 15 days, metrics get deleted, and if your boss asks “what was CPU at 3 PM two weeks ago?” you’re toast.
For small deployments (home labs, a 5-person startup, your friend’s SaaS) Prometheus is perfect. But the moment you start scaling (more metrics, longer retention, higher scrape frequency), you hit the wall. Hardware costs creep up. You look at your node and think: there has to be a better way.
Enter VictoriaMetrics. Same basic idea (time-series database), wildly different engineering underneath.
Which Metrics Backend Should Run Your Home Lab?
| Prometheus | VictoriaMetrics | |
|---|---|---|
| RAM for 10M series | 15 to 20 GB | 1 to 2 GB |
| Default retention | 15 days | Years, disk permitting |
| Ingestion rate | ~10K samples/sec | 100K+ samples/sec |
| Query language | PromQL | MetricsQL (superset) |
| Ecosystem maturity | Huge, 10,000+ dashboards | Smaller, growing fast |
| Cluster mode | Needs Thanos or Mimir | VMcluster built in |
What Is VictoriaMetrics, Really?
VictoriaMetrics is a time-series database built for efficiency. Not correctness: both Prometheus and VictoriaMetrics are correct, in the strict sense. But VictoriaMetrics optimizes for the opposite end of the tradeoff: it compresses like mad, uses way less RAM, and lets you retain years of data on the same hardware where Prometheus would choke after 15 days.
It comes in two flavors:
VMsingle: single-node mode. Drop-in replacement for Prometheus. One binary, one scrape config, one local database. You point your Grafana at it instead of Prometheus, and everything just works.
VMcluster: distributed mode. Multiple vmstorage nodes, vminsert nodes for ingestion, vmselect for queries. Scales horizontally if your metric cardinality explodes. But for most home labs? Overkill.
The magic is in the storage engine. VictoriaMetrics uses a columnar format optimized for time-series data. Prometheus stores metrics in a row-oriented WAL and in-memory index, great for hot queries, brutal for storage efficiency. VictoriaMetrics packs data differently: instead of “timestamp, value, timestamp, value,” it groups timestamps and values separately, then compresses each column hard. The result? Same metrics footprint can be 10x smaller on disk, and RAM usage drops because VictoriaMetrics doesn’t try to hold everything in memory.
Retention becomes cheap. Want 6 months of metrics? 1 year? VictoriaMetrics can do it on hardware that would make Prometheus weep. You’re trading sub-second latency for something more practical: fast-enough latency with sane resource usage.
MetricsQL vs PromQL: the Elephant in the Room
Here’s where things get spicy.
VictoriaMetrics ships with MetricsQL, which is almost PromQL, with extra stuff bolted on. It’s a superset: every PromQL query runs in MetricsQL. But MetricsQL has extra functions Prometheus doesn’t have, and some of the semantics are slightly different in ways that can surprise you.
For example, MetricsQL’s rate() function has an optional argument to specify the interval explicitly, where PromQL doesn’t. MetricsQL has histogram_quantile() improvements. It has alias() and label_set() for renaming, which is handy.
The gotcha? If you’re migrating dashboards and alert rules from Prometheus, 95% will work untouched. But that 5% that relies on quirky PromQL behavior (edge cases around instantaneous vector arithmetic, bool modifiers, group_left edge cases) might misbehave. It’s not a breaking change so much as a “you’ll spend a Saturday afternoon debugging” change.
Most people don’t notice. The extra functions are genuinely useful. But it’s a 99% drop-in that requires you to actually test your dashboards before flipping the switch.
vmagent: Scraping Without the Footprint
VictoriaMetrics also ships with vmagent, which is their answer to Prometheus’s scraper.
Prometheus is doing two jobs at once: scraping targets and storing metrics. VictoriaMetrics splits the concern. You can run vmagent wherever you want (tiny footprint, ~50 MB RAM), let it scrape your Kubernetes cluster or ten VMs, then push metrics to a central VictoriaMetrics instance via remote_write. Or run multiple vmagents and fan out to multiple VMsingle instances for redundancy.
Why care? Two reasons:
- Smaller footprint. vmagent is lean. Run it on a low-power edge device, a Raspberry Pi, inside Kubernetes as a DaemonSet. Prometheus would break a sweat; vmagent yawns.
- Fanout and HA. One vmagent can push metrics to multiple VictoriaMetrics backends. You can scrape once and replicate to three VMsingle instances for redundancy without hammering your targets.
You can actually run Prometheus with vmagent sending remote_write to VictoriaMetrics. Hybrid setup. Test the waters before committing.
The Other Pieces: vmalert, vmauth, vmui
VictoriaMetrics also bundles:
- vmalert: Alertmanager-compatible alert evaluator. It eats Prometheus alert rules (or MetricsQL versions), evaluates them against your VictoriaMetrics backend, and fires webhooks or sends to Alertmanager. One less component to run.
- vmauth: Reverse proxy for auth and multi-tenancy. If you’re sharing a VictoriaMetrics instance across teams, vmauth can enforce per-tenant isolation with basic auth or OAuth.
- vmui: Built-in Grafana-like UI for ad-hoc queries. Not as polished as Grafana, but it’s there. Handy for “quick check” debugging.
These are nice-to-haves. The core value is VMsingle + vmagent.
Benefits: the Real Wins
Memory efficiency. This is the headline. Prometheus with 10M time series = 15 to 20 GB RAM. VictoriaMetrics with the same 10M series = 1 to 2 GB RAM. Real numbers from real deployments. Your 2-core homelab box suddenly has headroom to run other stuff.
Retention. Want 2 years of metrics on 50 GB disk? VictoriaMetrics does it. Prometheus would burn that in a week.
Ingestion rate. Prometheus scrapes at ~10K samples/sec on modest hardware before sweating. VictoriaMetrics handles 100K+ samples/sec on the same box. Matters if you’re scraping densely instrumented Kubernetes clusters.
Compression. VictoriaMetrics compresses aggressively. Prometheus’s compression is minimal. Over weeks, the disk space difference is absurd.
Query speed. MetricsQL queries can be faster because VictoriaMetrics is optimized for columnar scans. Real dashboards see 2 to 5x query latency improvement.
The Tradeoffs: Maturity and Ecosystem
VictoriaMetrics isn’t Prometheus. It’s younger. The community is smaller. That means:
Dashboards. The Prometheus/Grafana ecosystem has 10,000 pre-built dashboards. “Kubernetes Cluster Monitoring,” boom, instant 30-panel dashboard. VictoriaMetrics has fewer drop-in dashboards. You’ll refactor a Prometheus one, which is tedious but not hard.
Integrations. Every observability tool on Earth speaks Prometheus. VictoriaMetrics is carving out space, but adoption is slower. That said, any tool that speaks Prometheus remote_write or the Prometheus HTTP API works with VictoriaMetrics.
Community size. Prometheus has broader mindshare. Easier to find blog posts, troubleshooting advice, example configs. VictoriaMetrics docs are actually excellent, but you’re not going to stumble on SO answers as easily.
Operator maturity. If you’re running Kubernetes, prometheus-operator is the standard. VictoriaMetrics has a community operator, but it’s less battle-tested. You might need to maintain YAML by hand.
None of this is a dealbreaker for a home lab or small prod cluster. It’s just: you’re swimming slightly upstream.
MetricsQL Gotchas (the 5%)
Let me be specific so you don’t spend Sunday debugging.
PromQL’s rate() and increase() work on raw counters. MetricsQL is more lenient, it can infer counter resets in some cases where Prometheus would return NaN. This usually means fewer false alerts, but it’s a behavior difference.
Aggregation operators (sum, avg, etc.) in PromQL group by label names explicitly. MetricsQL is more implicit. Edge cases around on() and group_left() might surprise you. Again, 99% of queries work; the 1% requires testing.
bool modifier (foo > 5 bool) works in PromQL but behaves differently in MetricsQL in some vector contexts. Real-world impact? Minimal. But if your alerting rules rely on it, test.
The fix: before migration, export your Prometheus alert rules and PromQL queries. Spin up a test VictoriaMetrics instance. Run your rules and queries against it. Find the oddities. Fix them. Then flip the switch in production.
Comparison to Mimir and Thanos
You might ask: “But wait, what about Grafana Mimir or Thanos?” Good question.
Mimir (formerly Cortex) is a long-term storage backend. You run it in Kubernetes, it’s horizontally scalable, it handles multi-tenancy, and it’s built for the cloud-native playbook. It’s excellent. Also: more complex than VictoriaMetrics. More knobs. More moving parts. If you’re already deep in Kubernetes and have ops people, Mimir is a solid choice.
Thanos is a sidecar/proxy that adds long-term object-storage backends (S3, GCS, etc.) to Prometheus. It doesn’t replace Prometheus; it extends it. You still run Prometheus locally for hot queries, then Thanos pulls old data from object storage. Great if you’re already on cloud infrastructure. Less useful for a single-node home lab.
VictoriaMetrics occupies a sweet spot: it is the storage engine. No Prometheus alongside it. No cloud dependency (though cloud integrations exist). Simpler than Mimir, more feature-complete than Thanos-as-an-add-on. Better fits the “I own my hardware” use case.
Migration Path: Run Both Side by Side
Here’s how to test the waters without gambling your monitoring:
- Deploy VictoriaMetrics (VMsingle, single binary).
- Configure your Prometheus to send
remote_writeto VictoriaMetrics:
remote_write: - url: "http://victoriametrics:8428/api/v1/write" queue_config: capacity: 100000 max_shards: 200 min_shards: 1Now Prometheus writes to both its local storage AND VictoriaMetrics. Both stay in sync. Point Grafana at VictoriaMetrics. Run dashboards for a week. If everything’s stable, flip Grafana back to Prometheus, fix any broken queries, then flip back to VictoriaMetrics. Once confident, disable Prometheus scraping. VictoriaMetrics takes over.
This is the zero-downtime path. Takes effort, but safe.
Docker Compose: Quick Start
Here’s a minimal setup to experiment:
services: victoriametrics: image: victoriametrics/victoria-metrics:latest container_name: victoriametrics ports: - "8428:8428" volumes: - vm_data:/victoria-metrics-data command: - "--storageDataPath=/victoria-metrics-data" - "--retentionPeriod=12" # 12 months retention - "--storage.maxDailySeries=0" # 0 = no daily churn limit restart: unless-stopped
vmagent: image: victoriametrics/vmagent:latest container_name: vmagent ports: - "8429:8429" volumes: - ./vmagent-config.yml:/etc/vmagent/config.yml - vmagent_data:/vmagent-data command: - "--promscrape.config=/etc/vmagent/config.yml" - "--remoteWrite.url=http://victoriametrics:8428/api/v1/write" depends_on: - victoriametrics restart: unless-stopped
volumes: vm_data: vmagent_data:Point vmagent to your existing Prometheus scrape_configs. It pushes metrics to VMsingle. Run Grafana against http://victoriametrics:8428. Done.
When the Switch Is Worth It
VictoriaMetrics isn’t a magic bullet. It’s a tradeoff. Make the switch when:
- You’re memory-constrained. Running Prometheus on a 4-core box with 8 GB RAM? You’re leaving metrics on the table. VictoriaMetrics gives you 2 to 3x more metric capacity on the same hardware.
- You need retention. 6+ months of data. Prometheus forces you to choose: buy bigger disks or delete old metrics. VictoriaMetrics makes long retention affordable.
- You’re ingesting high-cardinality data. Kubernetes with heavy container instrumentation? VictoriaMetrics handles it better.
- You’re migrating off Prometheus for cost. Every dollar saved on RAM and disk is a win.
Skip it if:
- You’ve got infinite ops budget. Run Mimir. No regrets.
- You’re locked into the Prometheus ecosystem. Heavy prometheus-operator users, tons of custom PromQL dashboards. Refactoring is tedious.
- You only need 15 days of metrics. Prometheus is fine. Less to learn.
For home lab and small production clusters? VictoriaMetrics is the move. It’s simpler than distributed systems (Mimir), cheaper than cloud (Cortex/Thanos), and gives you years of retention on hardware you already own.
Your 2 AM self will thank you when the disk isn’t full and your alerts are still firing.
