Qwen3.6-27B · vLLM · NVFP4 + MTP · conc 16

status: done

Configuration

ModelQwen/Qwen3.6-27B
CompanyAlibaba
FamilyQwen
Parameters27B (dense)
EnginevLLM + MTP (num_speculative_tokens=3) (speculative decoding)
Quant / precisionNVFP4
Why this quantconc-16 MTP point of the Qwen3.6-27B NVFP4 sweep — the base+MTP speedup-decay curve (EXPERIMENTS.md
Downloadunsloth/Qwen3.6-27B-NVFP4
Context window65536
Input modalitiestext, image, video (served text-only here)

Measured results

Prefill tok/s184.72
Decode tok/s192.23
Peak memory (GB)108.51 (system MemAvailable delta (10s sampling) — vLLM static KV reservation (util 0.85) + MTP head)
Completed2026-07-01 23:54 +0800

Full run command

# conc-16 MTP. Same recipe as -mtp-c8 — only --max-num-seqs differs.
scripts/bench-vllm-serving.sh unsloth/Qwen3.6-27B-NVFP4 65536 16 1000 600 256 \
  --trust-remote-code --dtype bfloat16 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

qwen3.6-27b Alibaba Qwen NVFP4 16-40B conc-16

Notes

conc-16 MTP point of the Qwen3.6-27B NVFP4 sweep — paired with the matched base -c16 for the base-vs-MTP speedup-decay curve (EXPERIMENTS.md #14), the last point before the published c32 row.

  • Result (conc 16): prefill 184.72 / decode 192.23 tok/s aggregate; 0 errors; peak mem 108.5 GB.
  • MTP speedup at conc-16: +64.9% (192.23 vs base 116.57) — and note MTP@c16 (192.2) already edges out base@c32 (187.7): the draft buys more than a 2× batch increase does. The full decay curve is now closed: +80.6 (c1) → +97.0 (c2) → +90.7 (c4) → +62.6 (c8) → +64.9 (c16) → +46.0 (c32)%. The dense 27B keeps a big MTP win far later than the 35B-A3B MoE (~+25–30%): it drops from the ~2× low-batch peak into a broad ~+60–65% plateau across c8–c16, then finally erodes toward +46% at c32 as the batch saturates compute.
  • Acceptance ~69%, accept-len ~3.08-of-3 (per-position 0.855/0.68/0.55) — flat across the whole sweep (c2 ~68% / c4 ~70% / c8 ~71% / c16 ~69%), confirming MTP acceptance is workload-driven, not concurrency- sensitive. The speedup decay is therefore pure batch-economics (shrinking spare compute), not draft quality.
  • Sweep: base -c16 · MTP -c1 · -c2 · -c4 · -c8 · c32.