A 6 GB VRAM laptop test: NVIDIA's Nemotron 3.5 30B scores 14/21 against Qwen 3.6 35B's 18/21

On the same RTX 3060 laptop (6 GB VRAM, about 64 GB RAM), NVIDIA's Nemotron 3.5 Lightning 30B-A3B took on Qwen 3.6 35B-A3B. On 21 fun-v1 capability tasks Nemotron passed 14 with 2 partial and 5 failures; Qwen passed 18 with 2 partial and 1 failure. llama-bench generation: 16.1 vs 19.2 tokens/s (3-round mean). Nemotron's Chinese commonsense results included a book-author mix-up and an invented source. One round per task — these results describe this single run.

Verdict

Loads
Yes

Runs when:The test machine had about 64 GB of RAM (peak system memory during generation was 43.8 GB; smaller configurations were not tested). Experts stay on the CPU via `--n-cpu-moe` (52 for Nemotron, 40 for Qwen), with remaining layers on the GPU via `-ngl 99`.

Main limits

  • Each capability task ran once; there is no variance.
  • G1 (snake game) and F1 (pelican on a bicycle) use check.py/Playwright pre-screening followed by human re-judgement of the recordings: Qwen's G1 was corrected from a false failure to 4/5 and F1 passed; the G8-zh1/zh3 human re-judgement can only lower scores.
  • Nemotron's experts were swept at 40/52/53 and 52 was chosen; Qwen stayed at ncmoe 40 and was not re-swept this episode.
  • Nemotron's 128K/256K numbers exist only from an earlier 10-22 reference run and were not re-tested the same day, so they are not used for comparison.
  • The English commonsense set also went 2 of 3 (the Holmes trap question invented a source) — the errors are not limited to Chinese.

Experiment

HardwareNVIDIA GeForce RTX 3060 Laptop GPU · 6 GB · laptop · CC 8.6
CPU / RAM12th Gen Intel(R) Core(TM) i7-12700H · 64 GB
ModelNemotron 3.5 Lightning 30B-A3B · UD-Q4_K_XL · moe
Frameworkllama.cpp · 623c16393
Tested2026-10-23T00:00:00+08:00

Video

Chinese哔哩哔哩 / BilibiliOpen on Bilibili
EnglishYouTubeOpen on YouTube

Results: 14 against 18 on 21 tasks

All 21 tasks come from the fun-v1 G/F groups, one round each:

Nemotron 3.5 30B Qwen 3.6 35B
Pass 14 18
Partial 2 2
Fail 5 1
Time for 21 tasks 2844 s (47.4 min) 1310 s (21.8 min)
Total session time incl. load 51.2 min 25.4 min
Model load 228 s 213 s
llama-bench generation, tg128 (3-round mean) 16.1 tok/s 19.2 tok/s
llama-bench prompt processing, pp512 (3-round mean) 228 tok/s 294 tok/s
16K prompt, first token 57.7 s (single run; earlier runs 66–83 s) 67.6 s
Generation at 32K depth (llama-bench, 3-round mean) 12 tok/s 15.2 tok/s
64K prompt, first token (single run) 280 s 297 s

The 16.1/19.2 and 228/294 figures come from llama-bench over three ABABAB rounds and are not chat speed. run-002 also has a benchmark set (Nemotron 19.64 from one round, Qwen 23.86 from three rounds; Nemotron's round was outside ±5%), not used here.

Nemotron's one clear win was an English logic question (F2-en): it thought for 185 seconds and answered correctly; Qwen answered in 51 seconds and got it wrong. The 16K first-token figure is a single capture, so it does not prove a consistent 16K win; the 64K figure comes from the run-003 server maxctx test.

Test conditions

  • Test date: 2026-10-23 (Beijing time).
  • Hardware: RTX 3060 Laptop 6 GB, i7-12700H, about 64 GB RAM (about 47 GB visible in WSL2).
  • Engine: llama.cpp 623c16393 (0.7.0-dev build 601); both models run with -ngl 99.
  • Models: NVIDIA-Nemotron-3.5-Lightning-30B-A3B-UD-Q4_K_XL (about 30B total, roughly 3B active per token); Qwen 3.6 35B-A3B UD-Q4_K_XL.
  • Speed and depth: run-003 llama-bench, three ABABAB rounds; the 16K first-token figure is a single run-004 capture.
  • The 21 tasks: run-004's fun-v1 set driven by Pi 1.4.0; sampling temperature 0.7 with a per-round seed; thinking configured as llama-server --reasoning off plus pi --thinking off.
  • G1 and F1 have no purely automatic grader: check.py/Playwright pre-screen, then a human re-judges the recording.

Where it loses

Chinese commonsense, 3 tasks: 1 partial and 2 failures. Asked for the four classic novels and their authors, Nemotron gave Luo Guanzhong for *Water Margin* and said it was also known as *Strange Tales from a Chinese Studio*. It placed the “waiting by a stump for a hare” story in *Huainanzi* instead of *Han Feizi*; for the “drawing feet on a snake” story it gave the wrong chapter of the *Strategies of Chu* and invented an ending. NVIDIA's official model card does not list Chinese among supported languages, which is consistent with this group.

English commonsense, 3 tasks: 2 passes and 1 failure. The failure was “Elementary, my dear Watson”: it claimed the line first appeared in 1915 in *The Adventure of the Crooked Man* — a story published in 1893 that does not contain the line.

The snake game failed all 5 checks. It wrote snake.html instead of index.html; the recording never leaves the Start Game screen and the console reports x is not defined.

The pelican task failed. The frame is essentially static — no visible legs, no recognizable bicycle. Qwen's version has spinning wheels and a pedalling foot.

Long-document needle tests passed on both. The needle was found at 10%, 50% and 90% in both the Chinese *Journey to the West* and the English Sherlock Holmes texts; Nemotron's slowest Chinese item took 148 seconds, while Qwen stayed around 80 seconds per item.

Reproducing it

# Speed (llama-bench, run-003, 3 ABABAB rounds)
llama-bench -m <Nemotron.gguf> -ngl 99 -ncmoe 52 -p 512 -n 128 -r 5
llama-bench -m <Qwen.gguf>     -ngl 99 -ncmoe 40 -p 512 -n 128 -r 5

# 64K first token (run-003 server maxctx): -c 65536 --temp 0
llama-server -m <Nemotron.gguf> -ngl 99 --n-cpu-moe 52 -c 65536 \
  --temp 0 -np 1 --jinja
llama-server -m <Qwen.gguf>     -ngl 99 --n-cpu-moe 40 -c 65536 \
  --temp 0 -np 1 --jinja

# 21 tasks (run-004): server without --temp; sampling injected by Pi
llama-server ... -ngl 99 --n-cpu-moe 52|40 -c 32768 -np 1 --jinja

Three caveats: the 16K / 32K server used -c 32768 while the 64K run used -c 65536; the 21 tasks used Pi-injected temperature 0.7, not the server's --temp 0; the table's speeds are llama-bench figures and must not be read as chat speed; run-002's CAP suite used the server's default sampling (temperature 1.0, random seed) and --thinking off did not actually silence reasoning, but that suite is not part of these 21 tasks.

Limits and sources

  • One round per task; the G group in particular only supports “on these questions”.
  • The partial results are G3 and G8-zh1 for Nemotron, and G3 and G1 for Qwen. G3 has no item-level evidence, G8-zh1 has a written re-judgement, and Qwen's G1 has 4/5 detail. G1 failed for Nemotron but was partial for Qwen; F1 failed for Nemotron and passed for Qwen.
  • About 64 GB is the test machine's configuration, not a measured minimum requirement.
  • Raw records: F:\AI\experiments\2026-10-22-nemotron-qwen-rerun\; summaries in 02-选题/nemotron35-30b/results-run-002-summary.md and results-summary-run004.md; model card: https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.

Evidence & revisions

run_idSourceClaim
NEM-001Tested hereevidence/results-summary-run004.md; evidence/results-nemotron.tsv; evidence/results-qwen.tsv; evidence/SCORING.md21 fun-v1 tasks: Nemotron 14 pass / 2 partial / 5 fail, Qwen 18 / 2 / 1; 2844 s vs 1310 s
NEM-002Tested hereevidence/speed-r{1,2,3}-ncmoe52.json; evidence/speed-r{1,2,3}-ncmoe40.jsonllama-bench 3-round means: tg128 16.1 / 19.2, pp512 228 / 294
NEM-003Tested hereevidence/ttft-16k-nemotron.json; evidence/ttft-16k-qwen.json16K prompt first token 57.7 s / 67.6 s (single capture)
NEM-004Tested hereevidence/depth-nemotron-r{1,2,3}.json; evidence/depth-qwen-r{1,2,3}.json32K-depth llama-bench generation 12 / 15.2 tok/s
NEM-005Tested hereevidence/results-summary-run004.md; evidence/results-nemotron.tsv; evidence/results-qwen.tsvChinese commonsense 1 partial + 2 fail; English 2 pass + 1 fail (invented source)

Revisions

  • 2026-11-09 —— First draft (English) added from the handover package; 2026-11-09 revised after independent review to fix speed labels, timing, task-set composition and grading description.