Promtail to Alloy Migration: A Practical Diff

Grafana Dropped the Deprecation Hammer

You logged into GitHub one day, checked the Promtail changelog, and saw it: “Promtail is now in security-only maintenance mode. Migrate to Grafana Alloy.”

Cool. Great. You have 20 Promtail configs across a dozen boxes. Three of them were written by a colleague who has since left the company. One of them has a pipeline stage you don’t fully understand but are afraid to touch because logs stopped breaking after you added it.

This is the migration guide that actually helps — not the one that says “just use alloy convert” and leaves you to figure out the rest when it errors on your custom stages.

What Even Is Alloy

Grafana Alloy is the successor to Grafana Agent in flow mode. It uses a configuration language called River (HCL-adjacent, not YAML) and it ships as a single binary that can replace Promtail, Agent, and more. The pitch is a unified telemetry pipeline — logs, metrics, traces — instead of running three separate agents.

For homelab and self-hosting use, you probably just want the logs part. That’s what we’re focusing on: a like-for-like swap of Promtail’s log scraping, pipeline stages, and Loki push.

Deprecation timeline, roughly:

• Mid-2024: Grafana starts steering users toward Alloy, Agent flow mode rebranded
• Mid-2025: Promtail officially in security-only maintenance (no new features, bug fixes stop)
• Late 2025 onward: Promtail still works but you’re on borrowed time

It still runs. It’ll keep running for a while. But the next Loki or Linux kernel change that breaks it? You’re on your own.

The Promtail Config We’re Migrating

Here’s a realistic Promtail config for a box running nginx, systemd services, and Docker containers:

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://loki:3100/loki/api/v1/push
    tenant_id: homelab

scrape_configs:
  - job_name: nginx
    static_configs:
      - targets:
          - localhost
        labels:
          job: nginx
          host: mybox
          __path__: /var/log/nginx/*.log
    pipeline_stages:
      - regex:
          expression: '(?P<method>\w+) (?P<path>[^\s]+) HTTP'
      - labels:
          method:
          path:
      - drop:
          expression: ".*healthcheck.*"
          drop_counter_reason: "dropped_healthchecks"

  - job_name: journal
    journal:
      max_age: 12h
      labels:
        job: systemd
        host: mybox
    relabel_configs:
      - source_labels: [__journal__systemd_unit]
        target_label: unit

  - job_name: docker
    docker_sd_configs:
      - host: unix:///var/run/docker.sock
        refresh_interval: 5s
    relabel_configs:
      - source_labels: [__meta_docker_container_name]
        regex: "/(.*)"
        target_label: container
      - source_labels: [__meta_docker_container_log_stream]
        target_label: stream
    pipeline_stages:
      - json:
          expressions:
            output: log
            stream: stream
      - output:
          source: output

Three jobs, a couple of pipeline stages, multi-tenant with tenant_id. Normal stuff.

The Equivalent Alloy Config

Alloy uses River syntax: components with blocks, not flat YAML. The mental model shift is from “config file” to “dataflow graph.” Each component declares its inputs and outputs explicitly.

// ── Loki write endpoint ─────────────────────────────────────────
loki.write "default" {
  endpoint {
    url = "http://loki:3100/loki/api/v1/push"
    headers = {
      "X-Scope-OrgID" = "homelab",
    }
  }
}

// ── nginx logs ──────────────────────────────────────────────────
loki.source.file "nginx" {
  targets = [
    {__path__ = "/var/log/nginx/*.log", job = "nginx", host = "mybox"},
  ]
  forward_to = [loki.process.nginx.receiver]
}

loki.process "nginx" {
  stage.regex {
    expression = `(?P<method>\w+) (?P<path>[^\s]+) HTTP`
  }

  stage.labels {
    values = {
      method = "",
      path   = "",
    }
  }

  stage.drop {
    expression = ".*healthcheck.*"
    drop_counter_reason = "dropped_healthchecks"
  }

  forward_to = [loki.write.default.receiver]
}

// ── systemd journal ─────────────────────────────────────────────
loki.source.journal "journal" {
  max_age       = "12h"
  forward_to    = [loki.process.journal.receiver]
  relabel_rules = loki.relabel.journal.rules
  labels        = {job = "systemd", host = "mybox"}
}

loki.relabel "journal" {
  rule {
    source_labels = ["__journal__systemd_unit"]
    target_label  = "unit"
  }
  forward_to = []
}

// ── Docker containers ────────────────────────────────────────────
discovery.docker "containers" {
  host             = "unix:///var/run/docker.sock"
  refresh_interval = "5s"
}

discovery.relabel "docker" {
  targets = discovery.docker.containers.targets

  rule {
    source_labels = ["__meta_docker_container_name"]
    regex         = "/(.*)"
    target_label  = "container"
  }

  rule {
    source_labels = ["__meta_docker_container_log_stream"]
    target_label  = "stream"
  }
}

loki.source.docker "docker" {
  host       = "unix:///var/run/docker.sock"
  targets    = discovery.relabel.docker.output
  forward_to = [loki.process.docker.receiver]
}

loki.process "docker" {
  stage.json {
    expressions = {
      output = "log",
      stream = "stream",
    }
  }

  stage.output {
    source = "output"
  }

  forward_to = [loki.write.default.receiver]
}

The key pattern: every source connects to a process pipeline connects to a write endpoint. Explicit wiring, no magic.

The alloy convert Command

Alloy ships with a conversion tool. Try it first — it handles the common cases:

alloy convert --source-format=promtail --output=config.alloy promtail.yaml

It’ll spit out a River config. For simple setups (static file scraping, basic regex stages), it’s pretty solid. Where it chokes:

• Custom match stages with complex pipelines nested inside — the converter flattens them incorrectly
• replace stages — not always translated cleanly
• multiline stages — hit or miss depending on configuration depth
• Anything using template stages — usually needs manual rewriting

If your pipeline has these, the converter output is a starting point, not a finish line. Run it, then diff against what you had and what you expect. Don’t trust it blindly.

After conversion, validate the syntax:

alloy fmt config.alloy           # formats in place, errors on bad syntax
alloy run config.alloy --dry-run # validates component wiring

Docker Compose Setup

Running Alloy on a single box with Compose. Mount your log directories and the socket:

services:
  alloy:
    image: grafana/alloy:latest
    container_name: alloy
    restart: unless-stopped
    volumes:
      - ./config.alloy:/etc/alloy/config.alloy:ro
      - /var/log:/var/log:ro
      - /var/lib/docker/containers:/var/lib/docker/containers:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - alloy_positions:/var/lib/alloy/positions
    ports:
      - "12345:12345"   # Alloy UI
    command:
      - run
      - /etc/alloy/config.alloy
      - --server.http.listen-addr=0.0.0.0:12345
      - --storage.path=/var/lib/alloy/positions

volumes:
  alloy_positions:

The Alloy UI at :12345 is genuinely useful — you can see component state, data flow rates, and whether your pipeline stages are matching anything. Promtail had /metrics and that was about it.

Full example: Clone the working files at github.com/KingPin/sumguy-examples/observability/promtail-to-alloy-migration

Label Cardinality: Don’t Make Loki Cry

This is where migrations go wrong. Promtail configs sometimes sneak in high-cardinality labels that Loki technically accepts but hates. The path label from the nginx pipeline above? Careful with that one.

If your nginx access log captures request paths and you label on them directly, you get a new log stream for every unique URL. That includes /api/v1/users/12345/profile, /api/v1/users/12346/profile, and so on forever. Loki becomes very unhappy. Your Loki operator becomes very unhappy. These are often the same person.

During migration, audit your labels:

# Check how many active streams you have in Loki right now
curl -s 'http://loki:3100/loki/api/v1/series?match[]={job="nginx"}' | \
  python3 -c "import sys,json; d=json.load(sys.stdin); print(len(d['data']))"

If that number is in the thousands and the job only has a few servers, something is fanning out. Find it before migrating or you’ll just be migrating a problem.

Better approach for path labels — either drop them entirely and use LogQL to filter by content, or use a replace stage to normalize paths:

stage.replace {
  expression = `/api/v1/users/[0-9]+/`
  replace    = `/api/v1/users/<id>/`
  source     = "path"
}

Multi-Tenant Loki: X-Scope-OrgID Still Works

If you’re running Loki in multi-tenant mode, the header approach in Alloy is clean:

loki.write "tenant_a" {
  endpoint {
    url = "http://loki:3100/loki/api/v1/push"
    headers = {
      "X-Scope-OrgID" = "tenant-a",
    }
  }
}

loki.write "tenant_b" {
  endpoint {
    url = "http://loki:3100/loki/api/v1/push"
    headers = {
      "X-Scope-OrgID" = "tenant-b",
    }
  }
}

Wire different source components to different write endpoints. It’s more verbose than Promtail’s tenant_id field, but it’s explicit — you can see exactly which pipeline sends to which tenant without hunting through config.

Rolling Rollout: Run Both, Compare Results

Don’t cold-cut from Promtail to Alloy. Run them in parallel for a few days:

1. Deploy Alloy alongside Promtail, both pushing to Loki
2. Temporarily add a alloy_source=true label in Alloy’s write config so you can distinguish streams
3. In Grafana, query both: {job="nginx", alloy_source="true"} vs {job="nginx"} (without the label — that’s Promtail)
4. Spot-check volume: are log counts roughly matching per time window?
5. Check pipeline stage output: are the extracted labels the same?

// Temporary label for migration comparison — remove after validation
loki.write "default" {
  endpoint {
    url = "http://loki:3100/loki/api/v1/push"
    headers = {
      "X-Scope-OrgID" = "homelab",
    }
  }
  external_labels = {
    alloy_source = "true",
  }
}

Once you’re satisfied the output matches, remove Promtail from the Compose stack and strip the alloy_source label from Alloy’s config.

Performance and Resource Usage

Alloy is roughly comparable to Promtail in memory for log-only workloads. On a box scraping a few hundred MB/day of logs, expect 50-150 MB RSS — same ballpark as Promtail.

Where Alloy pulls ahead:

• Introspection UI: port 12345 shows you live component state, throughput rates, errors. Debugging a broken pipeline stage takes minutes instead of log-spelunking
• Reloads without restart: some config changes can be applied via the API without restarting the agent
• Single binary for more things: if you later want to scrape metrics or traces from the same host, it’s already there

Where Promtail was simpler:

• YAML: love it or hate it, YAML is familiar. River has a learning curve
• Documentation: Promtail’s docs are more complete; Alloy’s are catching up
• Tooling: fewer community examples for Alloy yet — this is improving fast

The Bottom Line

Promtail still works. It’ll keep working until it doesn’t. The migration is not a weekend emergency, but it’s also not something to keep deferring.

The alloy convert command gets you 70-80% of the way there for typical configs. The rest is manual, but it’s also a good forcing function to audit your pipeline stages and label cardinality — stuff you probably should have looked at anyway.

The parallel-run approach is the right one. Run both for a week, compare results in Loki, retire Promtail when you’re confident. Your 2 AM self will thank you for having the Alloy UI when something breaks, instead of grepping through a container log to figure out why a regex stage stopped matching.

River syntax is weird for about a day, then it clicks. Explicit component wiring turns out to be pretty readable once you stop expecting YAML.

Start with one host. Get it right. Then roll it out to the rest.