Frigate Beyond Coral: GPU and OpenVINO

The Coral Ceiling

You’ve been running Frigate with a Coral USB accelerator for a year. It’s smooth. It’s cheap. You throw four cameras at it, detection runs in under 50ms per frame, and life is good.

Then you add the fifth camera. And the sixth. Suddenly you’re hitting 95% CPU load, detections are queuing, and that one persistent person standing near the driveway at 2 AM just became a ghost because the inference queue was full when they walked past.

Coral USB works until it doesn’t. And when it doesn’t, your options are: upgrade the accelerator hardware, move detection work to your GPU, or lean on OpenVINO with an iGPU and squeeze everything out of what you already have.

The internet sells you Coral as the home lab inference solution. It’s good marketing. But Coral maxes out around 15 to 20 FPS of concurrent detection work. If you’re running six cameras at 30 FPS apiece and trying to detect every frame, you’re asking a $60 stick to do $300 of work.

This is where GPU and OpenVINO come in.

The Real Numbers

Let’s talk performance before we talk config.

Coral USB (M.2): 4 TOPS, ~50 to 80ms per inference (full 320×320 model).

• Detections: 6 to 12 per second, serialized
• Bottleneck: PCI bus bandwidth (USB 3.0 is fine, but you’re still shuffling frames through the USB controller)
• Real-world throughput: ~15 FPS max if you’re pushing every frame through it
• Cost: $60 to 100

RTX 4060 (8GB VRAM, desktop GPU): 15+ TFLOPS, ~8 to 12ms per inference.

• Detections: 80+ per second, batched
• Bottleneck: VRAM (mostly a non-issue at 8GB)
• Real-world throughput: 40+ FPS concurrent per camera
• Cost: $200 to 250 (used), $300+ (new)
• Power draw: 70 to 100W (constant)

Intel Arc A770 (8GB VRAM, dGPU): 18 TFLOPS, ~7 to 10ms per inference.

• Detections: 90+ per second, batched
• Bottleneck: VRAM on the 8GB model (but fine for Frigate)
• Real-world throughput: 45+ FPS concurrent
• Cost: $150 to 200 (used)
• Power draw: 60 to 90W

OpenVINO + Intel iGPU (11th Gen or newer): ~6 to 8 TFLOPS shared, ~15 to 25ms per inference (depends on iGPU utilization).

• Detections: 40 to 60 per second
• Bottleneck: Shared memory bandwidth (you’re fighting your desktop for cycles)
• Real-world throughput: 25 to 35 FPS if your NVR host isn’t doing other GPU work
• Cost: free (hardware you already have)
• Power draw: 10 to 20W (iGPU adds minimal overhead)

The takeaway: Coral is the cheapest entry, but it’s a dead end. GPU means you’re buying more hardware. OpenVINO is the sweet spot if your NVR runs on a modern Intel iGPU (it’s Intel-only, AMD APUs need a different path).

GPU: The NVIDIA Path (Because Everyone Runs NVIDIA)

Here’s the config you need. I’m using an RTX 4060 because it’s 2026 and that’s what’s in used server auctions right now.

detectors:
  gpu:
    type: onnx
    device: 0  # which GPU (if you have multiple)

model:
  model_type: yolo-generic
  path: /config/yolov9-t-320.onnx
  input_tensor: nchw
  input_dtype: float
  width: 320
  height: 320

The detector type is onnx, not nvidia. There’s no nvidia detector type in Frigate. NVIDIA GPUs are picked up automatically by the ONNX runner as long as you run the -tensorrt image (more on that below). Frigate loads the ONNX model, shoves frames at the GPU, and comes back with detections.

But here’s where people get stuck.

You need:

• The NVIDIA Container Toolkit installed on the host (this is what used to be called nvidia-docker / NVIDIA Container Runtime)
• A recent NVIDIA driver: CUDA and cuDNN themselves ship inside the -tensorrt Frigate image, so you don’t install them on the host
• An ONNX model (not TensorFlow, not PyTorch weights)
• Enough VRAM (8GB is plenty for a 320×320 YOLO model; you only feel the squeeze if you’re running other GPU stuff alongside it)

Docker Compose with GPU support:

services:
  frigate:
    image: ghcr.io/blakeblackshear/frigate:stable-tensorrt
    container_name: frigate
    restart: unless-stopped

    volumes:
      - /path/to/config:/config
      - /path/to/storage:/media/frigate
      - /etc/localtime:/etc/localtime:ro

    environment:
      - YOLO_MODELS=yolov8n

    runtime: nvidia
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

    ports:
      - "5000:5000"
      - "1935:1935"

Key line: runtime: nvidia. Without that, Docker won’t see your GPU even if NVIDIA Container Runtime is installed.

The Gotchas

1. NVIDIA Container Runtime might not be the default.

Check:

docker info | grep nvidia

If it’s empty, you need to set it in /etc/docker/daemon.json:

{
  "runtimes": {
    "nvidia": {
      "path": "nvidia-container-runtime",
      "runtimeArgs": []
    }
  },
  "default-runtime": "nvidia"
}

Then sudo systemctl restart docker.

2. Precision is baked into the model, not a config toggle.

There’s no magic fp16: true line in the detector config. The TensorRT runner picks precision when it builds the inference engine from your ONNX model, and on RTX 30/40-series cards it’ll happily use FP16 tensor cores. If you want to force a specific precision, you do it when you export/convert the ONNX model, not in config.yml. Don’t go looking for a tf_lite or precision key in the detector block; they aren’t there.

3. Multiple processes will thrash your VRAM.

num_processes: 1 is the default for a reason. If you set it to 2 or 4, you’re not running four detectors in parallel: you’re running four copies of the model in VRAM. On an 8GB card with a 320×320 YOLOv8 model, that’s still fine. With multiple processes, you’ll page to system RAM and suddenly detection is slower than Coral.

4. Watch your power supply.

Adding a 4060 to a quiet NVR box means +70W constant. If you’re running this on a Ryzen 5600 system with a 400W PSU, you’re now at 85% load. That’s not wrong, but it means no headroom for spikes. Plan for 500W minimum if you’re adding a mid-range dGPU.

OpenVINO: The iGPU Path (Free Acceleration)

If you’re running a Proxmox or TrueNAS box on newer Intel hardware (12th Gen Alder Lake or newer), you already have an iGPU. Stick OpenVINO on it and you get 60 to 70% of the GPU performance for zero extra hardware.

detectors:
  openvino:
    type: openvino
    device: GPU.0  # Intel iGPU, slot 0
    num_processes: 1
    model:
      path: /path/to/models/yolov8n_openvino_model/

OpenVINO needs the model in its own format (.xml + .bin files). If you have an ONNX model, you convert it:

ovc yolov8n.onnx \
    --output_model ./yolov8n_openvino_model/yolov8n

(ovc is the OpenVINO Model Converter: it replaced the old mo Model Optimizer, which was fully removed after OpenVINO 2025.0. If you find a guide telling you to run mo --data_type FP32, it’s stale.)

But wait: does your Proxmox host even expose the iGPU to containers?

You need to pass the GPU device through:

services:
  frigate:
    image: ghcr.io/blakeblackshear/frigate:stable-openvino

    devices:
      - /dev/dri/renderD128:/dev/dri/renderD128
      - /dev/dri/card0:/dev/dri/card0

    volumes:
      - /path/to/config:/config

The exact device numbers (renderD128, card0) vary by system. Run ls -la /dev/dri/ to see what you have.

The Real Gotcha With OpenVINO

OpenVINO shares memory bandwidth with your NVR host’s main workload. If you’re running TrueNAS (heavy I/O) and Frigate (heavy GPU work) on the same iGPU, you’ll see:

• TrueNAS ZFS checksums thrashing L3 cache
• iGPU inference slowing down
• Frigate detections dropping from 30 FPS to 12 FPS during disk-heavy operations

The fix: pin Frigate to CPU cores and memory NUMA domains that are separate from your NAS workload. This is deeply system-specific and requires understanding your hardware topology (numactl -H is your friend).

Or, just accept that OpenVINO on a shared iGPU is “good enough” for 4 to 5 cameras, not a scaling solution.

Decision Tree: What Should You Actually Buy?

You have Coral and it’s not keeping up:

• 6+ cameras, 24/7 detection, < $200 budget: OpenVINO (if you have Intel iGPU). Switch to it, live with 25 to 30 FPS detection throughput.
• 6+ cameras, 24/7 detection, $200 to 400 budget: Used RTX 4060 or Arc A770. Spend $150 to 200, get 40+ FPS, never think about it again.
• 8+ cameras, heavy object classification (not just person/car): Budget for a GPU. Coral is done.
• Weird edge case (ARM NVR like Raspberry Pi 5, NVIDIA Jetson): Stick with Coral or jump to a Jetson with its onboard GPU. Don’t try to strap a dGPU to a Pi.

Real-World: Six Cameras, One RTX 4060

Setup:

• 6× 2MP cams at 30 FPS each = 360 FPS total video input
• RTX 4060, 8GB VRAM
• Frigate in LXC, GPU passed through via PCIe

Results:

• Inference time per frame: 9ms (compared to 65ms on Coral USB)
• Detection queue depth: stays under 5 frames
• CPU usage: 12% (GPU handling all the math)
• Detection accuracy: identical to Coral (same model)
• Power cost: ~$2/month extra electricity

Comparison to Coral USB on the same setup:

• Inference queue: 40 to 80 frames (frames aren’t being processed fast enough)
• CPU usage: 95% (Coral is slow, NVR is shoving frames at it, CPU has to wait)
• Drop rate on video playback: ~8% (can’t keep up with six simultaneous streams)

GPU wins. Not even close.

What About AMD and Older Intel?

AMD iGPU (Ryzen 5000G, 5700G): Here’s the catch nobody tells you: OpenVINO’s GPU plugin is Intel-only. It does not run on AMD GPUs, integrated or otherwise. If you’re on a Ryzen APU, your Frigate acceleration path is Frigate’s ROCm detector (newer, still rough around the edges) or just sticking a Coral M.2 in the board. Don’t expect OpenVINO to magically light up your Radeon graphics.

AMD dGPU (RX 7600, RX 6700): Same story. Frigate has a ROCm-based detector for these, but the ONNX/AMD path is finicky and driver-sensitive. Unless you already own the card, you’d be better off buying used NVIDIA or Intel Arc.

Intel 11th Gen or older iGPU: OpenVINO sees them, but performance is marginal. If you’re on Ice Lake Xeon from 2019, just buy a cheap Coral M.2 and call it a day.

Getting Your Hands Dirty

1. Check your GPU: nvidia-smi (NVIDIA) or clinfo (Intel/AMD)
2. Pick a detector type: GPU (NVIDIA) or OpenVINO (Intel iGPU)
3. Convert your model: ONNX for GPU, OpenVINO format for iGPU
4. Update your Frigate config (detectors section above)
5. Test on a single camera first: run 24 hours, check detection accuracy and queue depth
6. Expand to all cameras once you know it’s stable

And if you’re eyeballing the RTX 4060 route: you can find them in used enterprise auction lots for $140 to 170 right now. Don’t overpay new.

The Honest Take

Coral was the home lab inference gold standard for five years. But 2026 is different. Used GPUs are cheaper, faster, and more power-efficient per TFLOP than they were in 2021. If you’re building new Frigate infrastructure, skip Coral. If you’re upgrading, hit the used GPU market.

And if you’re on Intel and can’t stomach hardware shopping, OpenVINO is legitimately solid. Yes, you’ll squeeze out 30 FPS instead of 50, but it’s free, and your 2 AM self won’t have to troubleshoot a new piece of hardware that’s running hot in a tight case.

Pick your lane. Then stop second-guessing the detection queue.

Related Reading

• Reolink + Frigate: A Practical NVR Build