Qwen3.6-27B · vLLM · NVFP4 + MTP · conc 16
status: done
Configuration
| Model | Qwen/Qwen3.6-27B |
|---|---|
| Company | Alibaba |
| Family | Qwen |
| Parameters | 27B (dense) |
| Engine | vLLM + MTP (num_speculative_tokens=3) (speculative decoding) |
| Quant / precision | NVFP4 |
| Why this quant | conc-16 MTP point of the Qwen3.6-27B NVFP4 sweep — the base+MTP speedup-decay curve (EXPERIMENTS.md |
| Download | unsloth/Qwen3.6-27B-NVFP4 |
| Context window | 65536 |
| Input modalities | text, image, video (served text-only here) |
Measured results
| Prefill tok/s | 184.72 |
|---|---|
| Decode tok/s | 192.23 |
| Peak memory (GB) | 108.51 (system MemAvailable delta (10s sampling) — vLLM static KV reservation (util 0.85) + MTP head) |
| Completed | 2026-07-01 23:54 +0800 |
Full run command
# conc-16 MTP. Same recipe as -mtp-c8 — only --max-num-seqs differs.
scripts/bench-vllm-serving.sh unsloth/Qwen3.6-27B-NVFP4 65536 16 1000 600 256 \
--trust-remote-code --dtype bfloat16 \
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'
Notes
conc-16 MTP point of the Qwen3.6-27B NVFP4 sweep — paired with the matched base -c16
for the base-vs-MTP speedup-decay curve (EXPERIMENTS.md #14), the last point before the published c32 row.
- Result (conc 16): prefill 184.72 / decode 192.23 tok/s aggregate; 0 errors; peak mem 108.5 GB.
- MTP speedup at conc-16: +64.9% (192.23 vs base 116.57) — and note MTP@c16 (192.2) already edges out base@c32 (187.7): the draft buys more than a 2× batch increase does. The full decay curve is now closed: +80.6 (c1) → +97.0 (c2) → +90.7 (c4) → +62.6 (c8) → +64.9 (c16) → +46.0 (c32)%. The dense 27B keeps a big MTP win far later than the 35B-A3B MoE (~+25–30%): it drops from the ~2× low-batch peak into a broad ~+60–65% plateau across c8–c16, then finally erodes toward +46% at c32 as the batch saturates compute.
- Acceptance ~69%, accept-len ~3.08-of-3 (per-position 0.855/0.68/0.55) — flat across the whole sweep (c2 ~68% / c4 ~70% / c8 ~71% / c16 ~69%), confirming MTP acceptance is workload-driven, not concurrency- sensitive. The speedup decay is therefore pure batch-economics (shrinking spare compute), not draft quality.
- Sweep: base
-c16· MTP-c1·-c2·-c4·-c8·c32.