Spark-X2.5-4B on a 6 GB laptop: which model gets things done faster?

1. Conclusion: being fast and being right are not the same thing here

Two local models on one laptop. Spark-X2.5-4B fits entirely in 6 GB of VRAM. Qwen3.6-35B-A3B is an MoE: it hands 40 layers of expert computation to the CPU, and its weights live mostly in 64 GB of system memory. After 24 tasks and 72 formal runs, their strengths did not land in the same place.

On two of the three pre-registered main tasks both models got it right: reading a manual and answering three questions, and repairing a to-do page that loses its contents on refresh. On those two, Spark was much faster — a median of 7.17 s against 247.73 s on the manual task, and 23.32 s against 395.20 s on the to-do task. The third main task, tidying up orders, was a failure for both.

But across all 24 tasks scored per task, Qwen passed more: 8 for Spark against 16 for Qwen.

"2 completed" counts the three pre-registered main tasks only, with the task's allowed maximum of two revisions. Repeated runs do not add tasks, and a supplementary task that happened to pass once does not get folded into this number.

2. Test environment: how the two models were actually run

The Qwen model's 35B is the total parameter count; about 3B parameters are active per generated token. Its weights do not fit in 6 GB of VRAM, so -ncmoe 40 moves 40 layers of expert computation to the CPU and system memory feeds the weights — that is the main reason it is slower.

The two models are not at the same quantization level (Q4_K_M against UD-Q4_K_XL) and their architectures differ. This is a practical comparison on one machine, not a controlled quantization or parameter-count experiment. It does not say "4B beats 35B", and it does not say that 6 GB of VRAM holds a 35B model on its own.

3. Scoring: 24 tasks, per-turn budgets, and revision rules

The task set is frozen at laptop-task-v2: 24 English tasks, eight each in order and data tidying, manual question answering, and browser repair — 18 regular and 6 longer or harder stress tasks. Each model runs the set once, in a fixed task order, alternating which model goes first.

A first answer counts as a pass only if it is a complete output and every structural, factual, citation or browser-behaviour check passes. Partial correctness records partial checks but fails the task. A failed task gets at most two fixed feedback turns, and that feedback only says what failed — never the expected answer. A timeout, a dropped connection, or a truncated output ends that turn; a visible prefix cannot be scored as a pass.

The pre-registered main tasks DATA-001, DOC-005 and CODE-001 run five complete times per model, so that tasks cannot be picked after the fact. The other 21 run once each. That makes 48 first-round comparisons plus 24 repeats — 72 runs in total, all completed.

4. Main task 1 — manual Q&A: both correct, 34× apart on waiting time

Both models get the same roughly 7,500-word English manual and must answer three questions from the text alone, citing where each answer comes from. Both passed all five runs on the first answer, with the right values and the right citations.

The third question is the important one: neither model invented a figure.

The manual is fictional test material written for this experiment, not any vendor's real documentation. Every figure comes from the raw per-run records, not from video or screenshots. Times are the median from sending the request to finishing verification, excluding model load.

5. Main task 2 — to-do list persistence: both models fixed it

The task hands over a broken single-file to-do page: a newly added item disappears on refresh. The models must make adding, editing, checking off and deleting all persist, then open the page and verify it by actually operating it.

Both passed all five runs on the first answer, with no revisions. Verification is not a screenshot: it includes real clicks, a refresh, and reopening the page. It also covers the edge case of deleting the last item and refreshing into an empty list.

Task time includes the fixed save-and-verify pipeline used for both models; it is not the time a person would spend pasting and checking by hand.

6. Main task 3 — order tidying: neither model delivered completely

The task turns messy order information into a table: paid orders only, with products, quantities, shipping and totals checked line by line. One order, JB-101, should total $30.40.

Neither model delivered all six orders completely in any of the five runs. Spark finished sooner, but the amounts and the missing order still need a human check.

The seconds here are the time spent failing, not "time to get it right" — neither model has a passing run on this task, so there is no successful waiting time to report. A failure time only says how long it took to fail; it is not a performance result.

7. Supplementary task — citations must be complete

In a manual of roughly 16,000 words, the models must find regional rules: how long South-region tickets are kept, who receives archival requests, and what exception the HOLD state has. The rules to find: 45 days for ordinary tickets, 365 days for HOLD tickets, archival requests go to the GRAY team, approved by the archival owner; the manual does not state the cost of retrieving an archive, and neither model invented one.

Spark's problem here is incomplete citation. It did not misread the retention periods.

8. Supplementary task — deleting while filtered: Spark deleted the wrong item

The completed items are Alpha and Gamma; the unfinished one is Beta. Deleting Gamma from the "completed" filter view should leave Alpha and Beta.

The same "repair a to-do page" job separates the two models once a filter is involved. This is why passing the to-do main task does not let you assume this one passes too.

9. Supplementary tasks — invoice revision and cross-timezone totals

Invoice revision. The same invoice has several versions; only the highest revision counts, duplicates count once, and only paid entries are totalled. For IN-302 the newer version is paid, 5 × $7.20 with $2 shipping, and should count as $38.00 (the complete correct total for the task is $155.20).

Cross-timezone totals. Payments from different regions must be totalled by UTC date, with cancelled records excluded. One record, EV-01, is 23:30 local on 10 September at UTC−4, which in UTC is 03:30 on 11 September.

Qwen got the grand total of $43.50 right but grouped it on the wrong date. Checking only the final total would hide this.

10. Special notes: both models kept the text intact

Repairing a plain-text notes page: after saving, editing and refreshing, characters, spaces and line breaks must not change. Both models' first generated pages passed all four checks.

11. All 24 tasks: Qwen passes more

With the same revision allowance, here is how many tasks each model completed.

Spark has tasks it gets right faster; Qwen has more tasks it gets right. The two main tasks are where they agree. The real gap appears in citation completeness, and in the longer or harder tasks.

The denominator is the 24 different tasks, not 72: the five repeats of each main task do not add tasks. Each task allows at most two revisions and ends on timeout. Stress tasks are reported separately and never merged with the regular ones.

12. On this laptop, what I would pick

Try Spark first: long-document lookups and small web-page repairs. It got both main tasks right and waits far less. On small tasks whose result you can check quickly, it is the cheaper place to start.

Wait longer for Qwen when it matters: invoice revision, filtered deletion and exception citations were more complete, at dozens of times the waiting cost.

Double-check amounts and dates either way: order tidying and cross-timezone totals were never fully solved, so a neat-looking table still needs a line-by-line check.

These conclusions hold for this machine and these settings only. Beyond its 6 GB of VRAM this laptop has 64 GB of system memory; another computer or configuration needs its own test. All tasks are synthetic test material.

13. Reproducing it: full launch commands and shared settings

Apart from the MoE layer count, the two models were launched with identical parameters.

# Spark-X2.5-4B: all layers on the GPU, no experts offloaded to the CPU
llama-server -m Spark-X2.5-4B-Q4_K_M.gguf \
  --alias spark4b --host 127.0.0.1 --port 8081 \
  -c 32768 -t 20 -tb 20 -b 512 -ub 128 -ngl 99 \
  -fa on -ctk q8_0 -ctv q8_0 -np 1 \
  --jinja --reasoning-format deepseek --no-reasoning-preserve \
  # -c 32K context; -t/-tb 20 threads; -ngl 99 puts every layer on the GPU
  # -fa on FlashAttention; -ctk/-ctv q8_0 quantized KV cache; -np 1 single slot
  # --reasoning-format with --no-reasoning-preserve turns thinking output off

# Qwen3.6-35B-A3B: one extra flag, -ncmoe, moves 40 expert layers to the CPU
llama-server -m Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf \
  --alias qwen35b --host 127.0.0.1 --port 8082 \
  -c 32768 -t 20 -tb 20 -b 512 -ub 128 -ngl 99 \
  -fa on -ctk q8_0 -ctv q8_0 -np 1 \
  --jinja --reasoning-format deepseek --no-reasoning-preserve \
  -ncmoe 40
  # -ncmoe 40 is the only launch difference between the two models:
  # expert computation for 40 layers runs on the CPU

Both also ran with --fit off --no-context-shift --no-cache-prompt --cache-ram 0, which disables context shifting and prompt caching so that every turn recomputes the full history.

14. Appendix: the export-rules task — rules right, format revised once

This task asks for the export rules of version 3.1 and requires the exact codes the task specifies: a missing date must be written as the literal code NULL.

Spark passed first time: rules, source and the required format all correct. Qwen had the rules and the source right from the start, but wrote an empty value for the missing date on its first answer and passed after one revision.

Qwen did not misread the manual; what it had to fix was the answer format. This section is here to record a format requirement, not as an example of a factual error.