Turning thinking off was ~9× faster — and produced nine more wrong answers: Qwen 3.6 35B on a 6 GB VRAM laptop
One laptop, one Qwen3.6-35B-A3B, one 16-task set (E5 excluded after an ambiguous v1 grading rule; 15 tasks scored), three rounds — the only variable was whether thinking was enabled. With thinking on: 45/45 correct, 2772.7 s total. With it off: 36/45 correct, 295.5 s total. About 9× faster, with nine wrong answers. A simple strawberry question dropped from 19 s to 0.8 s; a discount trap that needed multi-step arithmetic went from 3/3 correct to 0/3.
Verdict
Runs when:Qwen3.6-35B-A3B `UD-Q4_K_XL` with llama.cpp `d854d98`, 32K context, KV f16, one fixed sampling set. Round 1 used its own server process; E5 v2, r2 and r3 shared another process (no restart between r2 and r3), changing only `enable_thinking`; build, model and launch flags stayed the same.
Main limits
- One model, one quantization, one machine.
- One of the 16 tasks (E5) had an ambiguous v1 grading rule (item_count should mean 3 item kinds; the model answered 15 total pieces), so both runs were marked PARTIAL in round 1. A v2 rule (item_kinds) passed, but E5 is excluded to keep the 15-task count consistent.
- The three rounds used different seeds (42 / 1001 / 2002); they still describe only those three rounds.
- The time saved is thinking time, not decode speed: median decode moved only from 21.7 to 24.9 tok/s (about 15%).
Experiment
| Hardware | NVIDIA GeForce RTX 3060 Laptop GPU · 6 GB · laptop · CC 8.6 |
|---|---|
| CPU / RAM | 12th Gen Intel(R) Core(TM) i7-12700H · 64 GB |
| Model | Qwen3.6-35B-A3B · UD-Q4_K_XL · moe |
| Framework | llama.cpp · d854d98 |
| Tested | 2026-11-07T08:36:00+08:00 |
Results: 9× faster, nine wrong answers
| Thinking on | Thinking off | |
|---|---|---|
| Correct | 45 / 45 | 36 / 45 |
| Total wall time | 2772.7 s | 295.5 s |
| Generated tokens | 56,380 | 3,034 |
| Median decode speed | 21.7 tok/s | 24.9 tok/s |
| E3 (count the r's in strawberry) | 19.0 s, answer 3 | 0.8 s, answer 3 |
| M1 (discount trap) | 3/3 correct, answer 317.3, median 64.0 s | 0/3 correct, answers 357.2 / 361 / 357.2, median 1.75 s |
| Tool-calling tasks | 9/9 correct, median 60.4 s | 9/9 correct, median 18.0 s |
M1 shows the sharpest contrast: with thinking off it answered in about 1.7 s — roughly 36× faster than the 54–65 s with thinking on — and was wrong all three times.
The nine failures with thinking off: M1 three times, M2 twice, H1 twice, M3 once, E4 once. Simple commonsense and tool calls barely changed; multi-step arithmetic and logic took the hit.
Test conditions
- Hardware: RTX 3060 Laptop, 6 GB VRAM, about 64 GB RAM (about 47 GB visible in WSL2).
- Model: Qwen3.6-35B-A3B
UD-Q4_K_XL; engine llama.cppd854d98(0.7.0-dev build 1). - Fixed settings: 32K context, KV f16 (default),
max_tokens16384, 900 s wall-clock cap per run,cache_prompt:false. - Sampling: temperature 0.7, top_p 0.8, top_k 20, min_p 0.0; identical for both modes.
- The only variable:
enable_thinkingin the request. Preflight confirmedreasoning_contentis empty when off and non-empty when on. - 16 tasks across commonsense, extraction, multi-step math/logic, code, hard reasoning, and tool calls; grading was scripted or manual against pre-written standards, never model self-assessment.
- Round differences: r1 used run_v1.py with no thermal cooldown; r2/r3 used run.py with a pre-request cooldown (r3's M1 waited 60 s); wall_s excludes cooldown time.
Where the time goes — and where the answers go
The speedup is mostly fewer tokens. Thinking on generated 56,380 tokens; off generated 3,034 — about 18.6×. Decode speed moved only from 21.7 to 24.9 tok/s, about 15%. What is saved is mainly the reasoning pass, not engine speed.
Easy tasks look identical. The strawberry count took 19.0 s with thinking and 0.8 s without; both answered 3.
Trap questions break. The discount-stacking M1 was 3/3 correct with thinking on (answer 317.3, 54–65 s per run). With thinking off it was 0/3 — answers 357.2 / 361 / 357.2 — in about 1.7 s: far faster, and wrong.
Tool calls are unaffected. All 9 passed both ways, with the median falling from 60.4 s to 18.0 s. That is the best trade in the set: fixed procedure, little judgement.
Reproducing it
# Server (r1 one process; E5 v2/r2/r3 share another; same flags)
llama-server -m <Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf> -c 32768 \
-ngl 99 --n-cpu-moe 99 --jinja --reasoning-format deepseek \
--port 8091 -np 1
# Request: change only this field
{ "chat_template_kwargs": { "enable_thinking": false } }Sampling fixed at temp 0.7 / top_p 0.8 / top_k 20 / min_p 0.0, cache_prompt:false, max_tokens 16384.
Preflight before running: with thinking off, reasoning_content must be empty; with it on, non-empty. Otherwise the variable is not controlled. Within a round, on/off order alternated by task; the same question was not guaranteed strict ABBA across rounds. Reproduce accordingly.
Limits and sources
- This is not “thinking off is always better”: M1 was about 36× faster and wrong all three times.
- The result covers one model at one quantization; re-test before generalising.
- Raw records:
F:\BotWork\experiments\thinking-on-off\—results_r1,results_r1_E5v2,results_r2,results_r3andcapture.log; a copy of the publishing bundle is in04-账号/publish/thinking-on-off/in/exp/. - The video publishes on 2026-11-13 at 20:00 (Bilibili) and 22:00 (YouTube); links will be added then.
Evidence & revisions
| run_id | Source | Claim |
|---|---|---|
| TOO-001 | Tested hereevidence/results_r1/runs.jsonl; evidence/results_r2/runs.jsonl; evidence/results_r3/runs.jsonl (E5 excluded from scoring) | 15 tasks × 3 rounds: thinking on 45/45, 2772.7 s; off 36/45, 295.5 s |
| TOO-002 | Tested hereevidence/results_r1/runs.jsonl; evidence/results_r2/runs.jsonl; evidence/results_r3/runs.jsonl; evidence/M1_on.json; evidence/M1_off.json | M1: on 3/3 correct (answer 317.3, median 64.0 s); off 0/3 (answers 357.2 / 361 / 357.2, about 1.7 s) |
| TOO-003 | Tested hereevidence/results_r1/runs.jsonl; evidence/results_r2/runs.jsonl; evidence/results_r3/runs.jsonl (T1–T3) | Tool calls 9/9 both ways, median 60.4 s / 18.0 s |
| TOO-004 | Tested hereevidence/results_r1/runs.jsonl; evidence/results_r2/runs.jsonl; evidence/results_r3/runs.jsonl (decode_tps and predicted_n) | Median decode 21.7 / 24.9 tok/s; generated 56,380 / 3,034 tokens |
| TOO-005 | Tested hereevidence/results_r1/runs.jsonl; evidence/results_r2/runs.jsonl; evidence/results_r3/runs.jsonl (E3) | E3 strawberry: 19.0 s on / 0.8 s off, both answer 3 |
Revisions
- 2026-11-09 —— First draft (English) added from the handover package; 2026-11-09 revised after independent review to fix the M1 answer/time mix-up, E5 scope, reproduce command and round handling.