Qwen3.6-27B · vLLM · NVFP4 + MTP · conc 4
status: done
Configuration
| Model | Qwen/Qwen3.6-27B |
|---|---|
| Company | Alibaba |
| Family | Qwen |
| Parameters | 27B (dense) |
| Engine | vLLM + MTP (num_speculative_tokens=3) (speculative decoding) |
| Quant / precision | NVFP4 |
| Why this quant | conc-4 MTP point of the Qwen3.6-27B NVFP4 sweep — the base+MTP speedup-decay curve (EXPERIMENTS.md |
| Download | unsloth/Qwen3.6-27B-NVFP4 |
| Context window | 65536 |
| Input modalities | text, image, video (served text-only here) |
Measured results
| Prefill tok/s | 68.39 |
|---|---|
| Decode tok/s | 67.76 |
| Peak memory (GB) | 108.06 (system MemAvailable delta (10s sampling) — vLLM static KV reservation (util 0.85) + MTP head) |
| Completed | 2026-07-01 23:35 +0800 |
Full run command
# conc-4 MTP. Same recipe as -mtp-c8 — only --max-num-seqs differs.
scripts/bench-vllm-serving.sh unsloth/Qwen3.6-27B-NVFP4 65536 4 1000 600 256 \
--trust-remote-code --dtype bfloat16 \
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'
Notes
conc-4 MTP point of the Qwen3.6-27B NVFP4 sweep — paired with the matched base -c4
for the base-vs-MTP speedup-decay curve (EXPERIMENTS.md #14).
- Result (conc 4): prefill 68.39 / decode 67.76 tok/s aggregate; 0 errors; peak mem 108.1 GB.
- MTP speedup at conc-4: +90.7% (67.76 vs base 35.54) — the dense 27B holds a near-2× MTP win right up through conc-4. The decay is now traced end to end: +80.6% (c1) → +97.0% (c2) → +90.7% (c4) → +62.6% (c8) — the win stays huge while the base is still bandwidth-bound (c1–c4) and only starts eroding once the batch begins filling the compute at c8, far above the 35B-A3B MoE’s ~+20–40% band.
- Acceptance ~70%, accept-len ~3.1-of-3 (per-position 0.86/0.70/0.55) — batch-stable vs the -c2 (~68%) and -c8 (~71%) rows; MTP acceptance here is workload-driven, not concurrency-sensitive.
- Sweep: base
-c4· MTP-c1·-c2·-c8·-c16·c32.