APEX + MTP on a 6 GB laptop: a 39% median generation gain, at what cost?

Here are the settings, individual task results and failures behind the video. Generation speed is not the same as end-to-end waiting time, and faster output does not guarantee a better answer.

1. Separate the weights from the MTP switch

After the previous video, a viewer recommended an APEX quantization with MTP support. I tested the same task set on the same laptop.

A→B changes both weights and quantization. B→C changes only MTP. The file is named I-Compact; the narration calls it iCompact. It comes from SC117's model repository, with the recorded revision 313e55f61a045dd18ac023d28fe845ee35988a5d.

2. Hardware and test conditions

There are 72 formal runs, 24 per group: one run for each of 12 tasks, plus four additional performance runs for DATA-004, DOC-001 and CODE-001 in each group. Each formal run starts a new server process, with loading measured separately. System file caches were not cleared, so this is not a cold-cache benchmark.

3. All 12 generation-speed comparisons

For each task, I divide C's generation speed by B's, then take the median of those 12 ratios. The result is about 1.390, or +39%. The equivalent B/A median is about 1.003, essentially flat.

Rows with n=5 use the median of five runs. Rows with n=1 are single measurements. Task-specific B→C gains range from roughly 11% to 63%. The table keeps two decimal places; the narration rounds these to 19.5→27.8 t/s for daily totals, 13.2→21.4 for the single long-document run, and 16.3→24.2 for the five-run webpage-repair median.

Answer lengths can differ across groups. To judge the wait for a complete answer, look at elapsed time as well; the 39% generation gain is not a claim that the entire request finishes 39% sooner.

4. Long inputs can hide the generation gain

A request first processes its input, then generates its answer. MTP mainly helps the second stage here. The retrieval task contains 12,869 input tokens but produces only a short line. Input processing therefore takes a large share of the wait, and faster generation makes less difference to the total.

16K identifies the task-set variant; 32K is the configured context window. Longer outputs, such as HTML, SVG and structured results, make the generation-stage difference easier to see.

5. Draft acceptance is not answer accuracy

This setup uses the model's own MTP head, with at most two draft tokens followed by main-model verification. Accepted drafts let one verification step advance more than one token. This is the draft-mtp path in the tested llama.cpp version.

In the original formal DATA-001-C log, 754 of 756 drafted tokens were accepted: 99.7%, averaging 2.99 tokens per step. The original candy run's acceptance rate was 77.7%.

Acceptance describes how many draft tokens the main model used. It is not task accuracy, and the speed gain still needs to be measured. A 99.7% acceptance rate does not mean a 99.7% speed increase.

6. VRAM and loading costs

The difference between C and B is 688 MiB, calculated from their separate median peaks. It is not a fixed increase for every task. The draft head and its cache need space; the observed draft cache uses F16 while the main-model KV cache uses q8_0.

The original A runs took about 3–26 seconds to load. Disk, filesystem and cache conditions are mixed into those figures, so I keep them as context rather than ranking the models by loading speed. This machine has 64 GB of RAM; the test does not establish a minimum RAM requirement.

7. Invoice revisions: inspect the result

This English-language task keeps the highest revision of each invoice, removes duplicate messages and includes only paid invoices. B and C passed. IN-302 keeps revision 2 with total_cents: 3800; the final grand_total_cents is 15520, in US cents as specified by the task.

Across the eight objectively scored first-answer tasks, A passed 5/8, B 6/8 and C 6/8. Performance repeats do not enlarge the quality denominator. This is not a claim that all 12 tasks passed.

8. Retrieval regression: A found it; B and C missed it

The retrieval task contains one true identifier and three decoys. In the formal results, A returned EQL-6238; B and C returned UNQ-2327, missing the true value. These actual terminal answers for the same task match the original R1 results.

All three groups passed the separate variable-tracking task. Each group ran each long-context task only once. The retrieval regression appears with the newer weights both with MTP off and with it on; it cannot be attributed to MTP alone.

9. Changed answers and failed animation

The candy answers are 36, 29 and 28 for A, B and C. Even with the same newer weights, changing the MTP switch changes the answer, so I cannot simply say MTP has little effect on the answer itself.

All three groups produced HTML/SVG for a pelican riding a bicycle, with C generating fastest. In the original animations, feet did not follow the pedals properly and wheels separated from the frame. None handled the animation well. This test does not establish that VRAM capacity caused those failures.

Generation speed and animation quality are separate judgments. The animation assessment here uses the original formal artifacts.

10. Webpage repair and settings to try

The repair prompt supplies a single-file page that loses data. Across five runs, median generation speed increased from 16.3 to 24.2 t/s with MTP on. These two screenshots are functional checks of the original C artifact, after adding an item and after reopening it.

These screenshots document the original artifact's behavior. Generation speed remains the median of five performance runs.

This WSL/Linux example is arranged from the tested settings. Use a llama.cpp build supporting draft-mtp and replace the binary and model paths. It is a configuration example, not an additional benchmark recording.

./llama-server \
  --model "/path/to/Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-Preserved-APEX-I-Compact.gguf" \
  --ctx-size 32768 --threads 10 --batch-size 512 --ubatch-size 128 \
  --n-gpu-layers 99 --n-cpu-moe 40 \
  --cache-type-k q8_0 --cache-type-v q8_0 --flash-attn on \
  --parallel 1 --no-cache-prompt --no-context-shift \
  --spec-type draft-mtp --spec-draft-n-max 2 \
  --host 127.0.0.1 --port 8090

For the same-weight B comparison, remove --spec-type draft-mtp --spec-draft-n-max 2 and keep the other conditions fixed. Thinking, sampling and request caching also need to be set in the request. Replace the example question below; 2048 is an example output budget, while formal tasks use their own budgets.

{
  "messages": [
    {
      "role": "user",
      "content": "Replace this text with your question."
    }
  ],
  "stream": true,
  "max_tokens": 2048,
  "temperature": 0,
  "top_p": 1,
  "top_k": 0,
  "min_p": 0,
  "seed": 20261008,
  "repeat_penalty": 1,
  "presence_penalty": 0,
  "frequency_penalty": 0,
  "cache_prompt": false,
  "chat_template_kwargs": {
    "enable_thinking": false,
    "preserve_thinking": false
  }
}

Save the JSON above as request.json, then send it from another terminal:

curl -N \
  -H 'Content-Type: application/json' \
  --data-binary @request.json \
  http://127.0.0.1:8090/v1/chat/completions

11. Is MTP worth enabling for your tasks?

For faster generation on this machine and configuration, I would try it: the median gain is 39%, with additional VRAM usage. The weights must retain MTP support, the configuration must be correct and enough VRAM must remain available. Check the answers on your own tasks too; acceptance is not a substitute for correctness.

Statistics on this page come from 72 formal runs on 2026-10-08. Terminal examples come from 18 actual supplemental runs for this episode, stored separately. Those recordings did not replace the formal speed, memory or quality results. Prompts and terminal text stay in their original languages; the English page adapts only the explanation around them.

The conclusion is limited to this machine, these weights, these settings and these tasks. It does not promise the same gain on every 6 GB device or establish that general capability never regresses. Model and version links are above; the task-level speeds and sample sizes are in section 3.