KUN Field Notes
Search中文

Build llama.cpp with CUDA on a Windows Laptop via WSL2 (RTX 3060)

Build llama.cpp with CUDA 12.8 in WSL2 on a Windows 11 laptop to get llama-server and llama-bench. Every 6GB test on this site uses this setup, and three settings are the easiest to get wrong.

KUN · Tested 2026-09-22 · Updated 2026-10-16

Environment

  • Windows 11, RTX 3060 Laptop 6GB, driver 610.47.
  • Ubuntu 24.04 in WSL2.
  • CUDA toolkit 12.8 inside WSL.
  • Official llama.cpp repo, commit 9655061 in our tests.

Make sure WSL can see the GPU first:

nvidia-smi

Only continue if the RTX 3060 Laptop is listed.

Toolchain

sudo apt install -y build-essential cmake ninja-build git aria2

Install the CUDA toolkit from the WSL-specific repo (wsl-ubuntu), not a package that bundles a driver; Windows provides the driver. On a cloud server, use the ubuntu2404 repo instead.

Build

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86
cmake --build build --config Release -j

Set CMAKE_CUDA_ARCHITECTURES from your GPU's compute capability. The RTX 3060 Laptop is 8.6, hence 86; copying 120 from a 50-series guide produces binaries that won't run. Check yours with nvidia-smi --query-gpu=compute_cap --format=csv.

Verify

./build/bin/llama-cli --list-devices

You should see CUDA0. If only the CPU is listed, CUDA isn't linked and any speeds you measure are meaningless.

Binaries end up in build/bin/: llama-server, llama-cli and llama-bench.

Three common mistakes

  1. Wrong arch: see above.
  2. Installing a driver inside WSL: don't.
  3. Models on /mnt/c or /mnt/f: cross-filesystem reads are slow. A 22GB model loaded from /mnt/f timed out on our first agent run. Keep models on the WSL filesystem.

Next

FAQ

What CMAKE_CUDA_ARCHITECTURES value? Use your GPU compute capability; RTX 3060 Laptop is 8.6, so 86.

How to confirm CUDA works? Run llama-cli --list-devices and look for CUDA0.

References