Synthetic Browser Monitoring with Playwright + Grafana

A 200 OK Doesn’t Mean Login Works

You’ve got a status page. It pings your application every 60 seconds and draws a nice green line in Grafana. Your app is up, you think. Everything’s fine. Go grab coffee.

Then at 3 AM, your users can’t log in. But the HTTP 200 kept flowing the whole time.

Welcome to the gap between “HTTP probe OK” and “actually working.” Your app might be returning 200 from a broken login endpoint, or the JavaScript that powers your auth form never loaded. Your API returns data, but the frontend is a blank white screen. Your WireGuard gateway answers pings, but the DNS resolver behind it is dead.

HTTP status codes are a lie detector that only works if you’re checking the right lie. Synthetic browser monitoring, running actual Playwright tests against your app like a user would, catches the stuff HTTP probes miss. A login form that renders but never submits. A dashboard that loads but doesn’t populate. A payment flow that breaks at step 3 of 5.

This is how you catch those at 2 PM instead of 3 AM.

Why HTTP Probes Aren’t Enough

A status code doesn’t care if your app works. It cares if the server is listening.

An HTTP GET to /health returns 200 if your Go binary is running and has a handler. It doesn’t know if your database connection pooled empty, or if the JavaScript that powers your SPA never loaded, or if your login endpoint accepted the request but never actually validated the credentials.

Single-page apps (React, Vue, Svelte, Angular) are especially sneaky. Your /index.html can return 200, the JavaScript bundle can load, and the browser can render something, but if there’s a bug in your fetch() call or a missing API endpoint, the user sees a blank screen or an infinite spinner.

Multi-step flows are even worse. An HTTP probe can hit the first endpoint, get a 200, and declare victory. But if step 2 (the login form submission) breaks, or step 3 (the permission check) fails, a user hitting the happy path never makes it through.

Commercial uptime checkers like Pingdom and Datadog Browser Tests know this. They run real browsers against your app every few minutes. They click buttons. They fill forms. They assert that elements exist on the page. They catch auth flows breaking, third-party script failures, and JS errors that HTTP can’t see.

The catch? You can do this yourself with Playwright. Open source. No SaaS bill. Full control.

Playwright as a Synthetic Monitor

Playwright is a headless browser automation library built by Microsoft. It drives Chromium, Firefox, or WebKit from Node.js or Python. Normally people use it for testing, but it’s also a perfect fit for synthetic monitoring.

Here’s what makes it good for this job:

Stable, assertion-heavy. Playwright’s assertion library (expect() in JS/TS) waits for elements to exist, text to appear, or values to match, with built-in retry logic. You don’t write flaky timing-based waits. You say “wait for the button to be visible and clickable” and Playwright handles the polling.

Headless by default. Run it in a container, in a CI/CD system, or as a cron job. No X server, no VNC, no desktop environment needed. Just Chromium in a 700 MB Docker image.

Trace and screenshot capture. When a test fails, Playwright can save a video, a trace file (interactive debugging in the browser), or screenshots of each step. At 3 AM when your dashboard is broken, you’ve got a video showing exactly where it failed instead of squinting at a log file.

Real browser behavior. It’s not mocking network or DOM. It’s running actual browser code, JavaScript, CSS, fetch calls, third-party scripts, everything. If your app breaks in production, Playwright will catch it the same way a user does.

Multi-browser coverage. A single test can run against Chrome, Firefox, and Safari. Different rendering engines, different quirks. If Safari broke your layout, Playwright will find it.

The downside? Tests are slower than HTTP probes. A Playwright test takes 10 to 30 seconds to execute. You’re not running checks every 10 seconds. You’re running them every 5 to 10 minutes, depending on your tolerance and test count.

Running Playwright as Scheduled Synthetic Tests

The basic flow:

1. Write a Playwright test that mirrors your user’s actual workflow (log in, navigate to the dashboard, check a critical element exists)
2. Run it from a container on a schedule (cron, Kubernetes CronJob, whatever)
3. Capture the outcome (success, failure, duration, errors) and export it somewhere that can alert you or graph it
4. If it fails, capture a video or trace for debugging

Let’s build this piece by piece.

Example: A Login + Dashboard Check

Here’s a Playwright test for a self-hosted app with login:

import { test, expect } from '@playwright/test';

test('login and load dashboard', async ({ page }) => {
  // Navigate to app
  await page.goto(process.env.APP_URL || 'http://localhost:3000', {
    waitUntil: 'networkidle'
  });

  // Fill login form
  await page.fill('input[name="username"]', process.env.LOGIN_USER);
  await page.fill('input[name="password"]', process.env.LOGIN_PASS);

  // Submit and wait for redirect
  await page.click('button[type="submit"]');
  await page.waitForURL('**/dashboard**', { timeout: 10000 });

  // Assert critical content loaded
  const stats = page.locator('[data-testid="dashboard-stats"]');
  await expect(stats).toBeVisible({ timeout: 5000 });

  // Optionally check for JS errors during the flow
  const errors: string[] = [];
  page.on('console', msg => {
    if (msg.type() === 'error') errors.push(msg.text());
  });

  if (errors.length > 0) {
    throw new Error(`Console errors during test: ${errors.join(', ')}`);
  }
});

This test:

• Navigates to your app and waits for the page to settle (networkidle)
• Fills the login form from environment variables (never hardcode credentials)
• Submits and waits for a redirect to the dashboard
• Asserts that a specific dashboard element is visible within 5 seconds
• Fails if console errors appear (catches broken JS silently returning)

Docker Compose: Running Tests on a Schedule

Here’s a minimal setup:

version: '3.8'

services:
  playwright-monitor:
    image: mcr.microsoft.com/playwright:v1.61.0-noble
    environment:
      APP_URL: http://app:3000
      LOGIN_USER: [email protected]
      LOGIN_PASS: ${MONITOR_PASSWORD}
      PROMETHEUS_PUSHGATEWAY: http://pushgateway:9091
    volumes:
      - ./tests:/app/tests
      - ./results:/app/results
    working_dir: /app
    entrypoint: /app/entrypoint.sh
    restart: unless-stopped

  pushgateway:
    image: prom/pushgateway:latest
    ports:
      - "9091:9091"
    volumes:
      - pushgateway-data:/pushgateway
    command:
      - "--persistence.file=/pushgateway/metrics"

  prometheus:
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus-data:/prometheus
    command:
      - "--config.file=/etc/prometheus/prometheus.yml"

volumes:
  pushgateway-data:
  prometheus-data:

The entrypoint script runs Playwright and exports the result:

#!/bin/bash
# Note: no `set -e` here — we WANT the script to keep going when the test
# fails so we can push a failure metric. Bailing out early would mean a broken
# login never shows up in Grafana, which defeats the whole point.
set -o pipefail

cd /app/tests

# Run the test with trace capture on failure
npx playwright test --reporter=list 2>&1 | tee /app/results/output.log

# Capture the test's exit code. pipefail makes $? reflect playwright, not tee.
EXIT_CODE=$?

# Export metrics to Pushgateway
if [ $EXIT_CODE -eq 0 ]; then
  RESULT=1
else
  RESULT=0
fi

# PROMETHEUS_PUSHGATEWAY already includes the scheme (http://pushgateway:9091),
# so don't prefix another http:// here or curl will choke on http://http://...
cat <<EOF | curl --data-binary @- ${PROMETHEUS_PUSHGATEWAY}/metrics/job/playwright-synthetic/instance/app
# HELP playwright_test_success Boolean: 1 if test passed, 0 if failed
# TYPE playwright_test_success gauge
playwright_test_success{app="app"} ${RESULT}
# HELP playwright_test_duration_seconds How long the test took
# TYPE playwright_test_duration_seconds gauge
playwright_test_duration_seconds{app="app"} $(grep -oP '(?<=in )\d+' /app/results/output.log || echo 0)
EOF

exit $EXIT_CODE

Run this container on a schedule. If you’re using Kubernetes:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: playwright-synthetic-monitor
spec:
  schedule: "*/5 * * * *"  # Every 5 minutes
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: playwright
            image: mcr.microsoft.com/playwright:v1.61.0-noble
            env:
            - name: APP_URL
              value: http://app:3000
            - name: LOGIN_USER
              valueFrom:
                secretKeyRef:
                  name: monitor-creds
                  key: username
            - name: LOGIN_PASS
              valueFrom:
                secretKeyRef:
                  name: monitor-creds
                  key: password
            volumeMounts:
            - name: tests
              mountPath: /app/tests
            - name: entrypoint
              mountPath: /app/entrypoint.sh
              subPath: entrypoint.sh
          volumes:
          - name: tests
            configMap:
              name: playwright-tests
          - name: entrypoint
            configMap:
              name: entrypoint-script
          restartPolicy: Never

Secrets stay in Kubernetes secrets. Credentials never touch the image.

Exporting Results to Prometheus + Grafana

Two approaches, both solid:

Pushgateway approach (above): Your test script pushes a metric to the Prometheus Pushgateway after each run. Prometheus scrapes the gateway periodically. Simple, fire-and-forget, no need for the test container to expose a metrics port.

Textfile exporter approach: Write metrics to a file, and a separate exporter (like node_exporter) reads and exposes them to Prometheus:

cat > /var/lib/node_exporter/textfile_collector/playwright.prom <<EOF
# HELP playwright_test_success 1 if test passed
# TYPE playwright_test_success gauge
playwright_test_success{test="login_flow"} 1
playwright_test_duration_seconds{test="login_flow"} 12.5
playwright_test_timestamp{test="login_flow"} $(date +%s)
EOF

Then in node_exporter, point the --collector.textfile.directory flag at that directory.

Grafana k6 approach: If you want load testing + synthetic monitoring together, Grafana’s k6 (formerly Load Impact) has a Playwright integration and pushes results to Grafana Cloud. Single pane of glass: latency, errors, and real-browser test outcomes. This is the SaaS-flavored cousin, you get hosted dashboards, less ops overhead, but you’re sharing infrastructure.

For self-hosted, stick with Prometheus + Pushgateway or textfile collector. You own the data, no SaaS bill, and Prometheus is already in your stack if you’re doing monitoring seriously.

Grafana Dashboard

Create a dashboard with two panels:

Status (gauge): playwright_test_success, shows 1 (green) or 0 (red). Alerting rule: if this drops to 0 for 2+ minutes, page oncall.

alert: PlaywrightTestFailing
expr: playwright_test_success < 1
for: 2m
annotations:
  summary: "Synthetic login test is failing"
  runbook: "Check /results directory on the monitor container for video/trace"

Duration (line graph): playwright_test_duration_seconds, watch for creeping latency. If your test usually takes 8 seconds and suddenly it’s 22 seconds, something’s slow. Not a hard alert, but a canary for degradation.

Comparing to Commercial Synthetic Monitors

Datadog Browser Tests: Same concept, hosted version. ~$0.10 per test run, multiple geographic locations, beautiful integrations with Datadog dashboards. Vendor lock-in. You’re paying for their infrastructure and scale.

Pingdom RUM + Uptime: Similar, runs real browsers, includes real user monitoring. More expensive per test, but integrated monitoring suite.

Playwright DIY: You host it. You pay for the container CPU and the Prometheus disk. At scale (hundreds of tests), self-hosting gets cheaper. You own the tests in git. You control exactly what’s being tested and when.

The tradeoff: Datadog handles geo-distributed execution for you (test from 20 cities). DIY means you run it from one location (your lab, your server, your Kubernetes cluster). If you need geographically distributed checks, you either run multiple instances (one in AWS, one in DigitalOcean, one on-prem) or accept single-point-of-presence testing.

For most home labs and self-hosted apps, single-location testing is fine. You’re monitoring your own infrastructure, not a global SaaS product.

Avoiding Common Pitfalls

Hardcoding credentials: Never put passwords in the test file or Dockerfile. Use environment variables, injected from secrets management (Kubernetes secrets, HashiCorp Vault, your env file). The test container should not be buildable with credentials already baked in.

Timing-based assertions: Avoid page.waitForTimeout(2000) and then asserting. Use Playwright’s built-in waits:

// Bad
await page.waitForTimeout(2000);
await expect(page.locator('.dashboard')).toBeVisible();

// Good
await expect(page.locator('.dashboard')).toBeVisible({ timeout: 5000 });

The second waits up to 5 seconds for the element, polling every 100ms. The first always waits exactly 2 seconds, even if the element appeared after 200ms. Bad for flakiness.

Animation flake: If your app has fade-ins or transitions, they might interfere with assertions. Turn off animations in test mode:

// In your app config or test setup
if (process.env.TEST_MODE) {
  document.documentElement.style.setProperty('--animation-duration', '0s');
}

Or Playwright has a @media (prefers-reduced-motion: reduce) hook you can leverage.

Not capturing failures: If a test fails, you’ve got nothing but a boolean. Capture a trace:

test('login flow', async ({ page, context }) => {
  const tracing = context.tracing;
  await tracing.start({ screenshots: true, snapshots: true });

  // ... test code ...

  if (testFailed) {
    await tracing.stop({ path: '/results/trace-failed.zip' });
  } else {
    await tracing.stop();
  }
});

Download the trace file and open it in the Playwright Inspector to step through the failure frame-by-frame.

Secrets in traces: Playwright traces might capture sensitive data (API keys in headers, passwords in form fields). Sanitize before uploading:

page.on('request', (request) => {
  const headers = request.headers(); // synchronous; allHeaders() returns a Promise
  if (headers.authorization) {
    headers.authorization = '[REDACTED]';
  }
});

When Synthetic Browser Tests Are Worth It

HTTP probes catch downtime. Synthetic browser tests catch broken-ness.

Use them when:

• Your app is a SPA (React, Vue, Angular): HTTP probes can’t see JS rendering failures
• Auth is critical: login flows, 2FA, session validation need to actually work
• Multi-step workflows matter: payment flows, onboarding, provisioning where step 2 depends on step 1
• You run a self-hosted app and want to catch issues before users report them
• You’re paranoid about JavaScript errors silently breaking the UI

Skip them if:

• Your app is API-only and you’re just monitoring endpoints
• You’re OK with “HTTP 200 = up, everything else = down”
• Your test infrastructure cost isn’t justified by the stakes of downtime (a hobby project running one app)

For most self-hosted setups, Nextcloud, Wiki.js, your internal dashboard, your Grafana, one or two Playwright tests running every 5 minutes costs almost nothing (a few CPU cycles on your existing Prometheus machine) and buys you peace of mind. You’ll catch auth breakage, JavaScript errors, and CSS disasters hours before your users file tickets.

Setup takes an afternoon. The payoff is catching broken logins at 2 PM instead of 3 AM. That’s worth it.

Related Reading

• Blackbox Exporter: HTTP/TCP/DNS/ICMP Probes
• cAdvisor + Prometheus: Per-Container Metrics Done Right
• Glances vs Netdata: Two Free-Tier Monitors Compared
• Grafana Alloy: Replacing the Agent After Deprecation
• Grafana OnCall + Webhooks: Paging Without PagerDuty