Skip to content
Go back

LLMs on Intel Arc: SYCL vs Vulkan

By KingPin 10 min read
LLMs on Intel Arc: SYCL vs Vulkan
Contents

The Cheap 12 GB Card Nobody Told You How to Use

You priced out VRAM, noticed that an Intel Arc card gives you a lot of it for not much money, and then searched for how to run an LLM on it. Now you have six tutorials open. Half of them tell you to install IPEX-LLM. The other half tell you to use Vulkan. A third group says SYCL, with a build that needs a compiler you have never heard of.

Here is the short version. Use llama.cpp. Pick one of two backends: SYCL (Intel’s path) or Vulkan (the generic path). Build both if you can, run llama-bench on your own card with your own model, and keep the faster one. Do not install IPEX-LLM: Intel archived it on January 28, 2026, and the repository is read-only.

The rest of this post shows how to get each backend running, what each one costs you in setup, and how to measure them without trusting anyone’s blog numbers (including mine, which is why this post has none).

Why You Cannot Just Trust the Benchmarks

Search for “Arc B580 llama.cpp tokens per second” and you will find numbers for similar models that disagree by a wide margin. The causes are mundane:

So any ranking you read is a snapshot of one machine on one day. The method matters more than the number. We will get to the method after the setup.

Option 1: Vulkan (The Easy One)

Vulkan needs no vendor toolkit. The Intel driver already ships a Vulkan implementation through Mesa on Linux, so llama.cpp only needs the Vulkan development packages and a shader compiler.

Check that the card shows up:

Terminal window
sudo apt install vulkan-tools mesa-vulkan-drivers
vulkaninfo --summary

You want to see your Arc card listed as a physical device. If you only see llvmpipe, you are looking at a CPU software renderer and the driver is not loaded.

Build llama.cpp with the Vulkan backend (the build flag comes from the llama.cpp build docs):

Terminal window
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build-vulkan -DGGML_VULKAN=ON
cmake --build build-vulkan --config Release -j

You also need the libvulkan-dev, glslc, and spirv-headers packages (or the equivalent for your distro) before CMake succeeds.

Prefer not to build? The project publishes container images with Vulkan compiled in: ghcr.io/ggml-org/llama.cpp:server-vulkan, also light-vulkan and full-vulkan. The project notes that the GPU images are only built by CI, not tested on hardware, so treat them as convenient rather than verified.

Terminal window
docker run --device /dev/dri/renderD128:/dev/dri/renderD128 \
--device /dev/dri/card0:/dev/dri/card0 \
-v /srv/models:/models -p 8080:8080 \
ghcr.io/ggml-org/llama.cpp:server-vulkan \
-m /models/model-q4_k_m.gguf --host 0.0.0.0 --port 8080 --n-gpu-layers 99

The /dev/dri device mappings follow the same pattern the llama.cpp docs use for their SYCL images. Run ls -la /dev/dri first and pick the render node that belongs to the Arc card, not the integrated GPU.

Option 2: SYCL (The Intel One)

SYCL is the backend the llama.cpp docs treat as the Intel path. The support table lists the Arc A-series (A770, A750, A730M) and the Arc B-series (B580) as supported. The price is a heavier toolchain.

Three things have to be in place, in this order.

  1. GPU driver. Install Intel’s compute runtime from the Intel driver docs for your distro. Then add yourself to the render and video groups and log out and back in:

    Terminal window
    sudo usermod -aG render $USER
    sudo usermod -aG video $USER
  2. oneAPI toolkit. The build needs Intel’s icx and icpx compilers. Install the oneAPI Base Toolkit (the default location is /opt/intel/oneapi). The llama.cpp docs also point to the intel/deep-learning-essentials container if you do not want to install it on the host.

  3. Check that SYCL sees the card.

    Terminal window
    source /opt/intel/oneapi/setvars.sh
    sycl-ls

    You want a line like [level_zero:gpu] naming your Arc card. If the card is missing here, no amount of CMake will fix it. Go back to the driver.

Then build:

Terminal window
source /opt/intel/oneapi/setvars.sh
cmake -B build-sycl -DGGML_SYCL=ON \
-DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx \
-DGGML_SYCL_F16=ON
cmake --build build-sycl --config Release -j

GGML_SYCL_F16=ON builds the FP16 code path. The docs recommend it for better prompt processing in most cases and tell you to test FP32 as well, because the best choice varies by model. That means the F16 flag is a variable you should benchmark, not a setting to copy.

Run it on one GPU. If the machine has an integrated GPU too, SYCL may see both, and you do not want half your model on the slow one:

Terminal window
export ONEAPI_DEVICE_SELECTOR="level_zero:0"
ZES_ENABLE_SYSMAN=1 ./build-sycl/bin/llama-server \
-m models/model-q4_k_m.gguf -ngl 99 -sm none -mg 0 --port 8080

The ZES_ENABLE_SYSMAN=1 variable and the -sm none -mg 0 pair (no split, main GPU 0) come straight from the SYCL docs’ single-device example. The docs list a table of environment variables too, among them GGML_SYCL_ENABLE_FLASH_ATTN (on by default) and build options such as GGML_SYCL_SUPPORT_XMX, which controls the code paths that use Intel’s matrix instructions on cards that have them.

There is also a SYCL container route: the server-intel image from the llama.cpp registry, or a local build from .devops/intel.Dockerfile. It takes the same --device /dev/dri/... flags as the Vulkan example above.

Measure It Yourself with llama-bench

llama-bench ships with llama.cpp and prints prompt processing (pp) and token generation (tg) rates for a model. Run the same command with each build and put the results next to each other:

Terminal window
MODEL=models/model-q4_k_m.gguf
./build-vulkan/bin/llama-bench -m $MODEL -ngl 99 -p 512 -n 128 -r 5
ONEAPI_DEVICE_SELECTOR="level_zero:0" \
./build-sycl/bin/llama-bench -m $MODEL -ngl 99 -p 512 -n 128 -r 5

Rules that keep the comparison honest:

Then decide per workload. A reasonable result is “Vulkan for the chat box, SYCL for the batch job,” and nothing stops you from keeping both binaries.

Why Not IPEX-LLM and Ollama?

Most older Arc tutorials lean on IPEX-LLM, Intel’s PyTorch-based library, and its Ollama integration. Two problems make that a bad starting point now:

  1. The intel/ipex-llm repository was archived on January 28, 2026. It is read-only. Its README says Intel will not provide development, support, or maintenance, and that the project has known security issues.
  2. The Ollama builds those guides describe are frozen at whatever version was current when the project stopped, so you do not get new models or fixes from upstream.

If you already have a working IPEX-LLM Ollama container, it keeps working. Just do not start a new setup on it. llama.cpp’s own llama-server speaks an OpenAI-compatible API, so most tools that point at Ollama can point at it with a changed base URL.

The llama.cpp build docs also list an OpenVINO backend for Intel GPUs and NPUs, and publish server-openvino images. I did not test it for this post, so treat it as a third option to try, not a recommendation.

What Fits in 12 GB

VRAM decides what you can run, and the speed question only matters once the model fits. A 12 GB card holds a 7B to 14B model at 4-bit quantization with room for context. Anything larger spills to system RAM and slows sharply. The same math applies to every vendor, so the GPU memory math post covers how to estimate weights plus KV cache before you download 20 GB of model.

The SumGuy Take

For a new Arc owner: start with Vulkan, because it needs no vendor toolkit and tells you whether the card is healthy. Then build SYCL if you want to squeeze more out of it, and let llama-bench settle the argument. Skip IPEX-LLM entirely.

An Arc card is a fine budget inference GPU if you accept that you are the integrator. Nobody will hand you a one-click installer, and the fastest backend this month may be the slower one after the next oneAPI release. That is the price of cheap VRAM, and for plenty of home labs it is a fair trade.

Common Questions

Does Ollama work on Intel Arc GPUs?

Not through a maintained Intel build. The IPEX-LLM project that provided Ollama support for Arc was archived on January 28, 2026. Run llama.cpp’s llama-server with the Vulkan or SYCL backend instead. It exposes an OpenAI-compatible API that most Ollama-targeting tools accept.

Is SYCL or Vulkan faster on Intel Arc?

Neither wins everywhere. Community reports split by card and by software version, and a single oneAPI toolkit update changed prompt processing on an Arc B570 by 3.1x in the llama.cpp docs. Build both backends, run llama-bench on your model with the same settings, and compare the pp and tg columns. That is a short job and beats any blog post, including this one.

Do I need to install the whole oneAPI toolkit?

Only to build the SYCL backend. The icx and icpx compilers come from oneAPI, and the llama.cpp docs also offer the intel/deep-learning-essentials container so the host stays clean. The Vulkan backend needs only the Vulkan development packages and a shader compiler such as glslc.

Why does sycl-ls not show my Arc card?

Usually the driver or group membership is wrong. Confirm that clinfo lists the GPU, that your user is in the render and video groups (and you logged out and back in), and that you ran source /opt/intel/oneapi/setvars.sh in the current shell. Fix the driver first. The llama.cpp build cannot see a device that the SYCL runtime cannot see.


Share this post on:

Send a Webmention

Written about this post on your own site? Send a webmention and it'll show up above once verified.


Next Post
Steam Deck Emulation With EmuDeck + NAS

Discussion

Powered by Garrul . Sign in with GitHub or Google, or post anonymously.

Related Posts