Qwen3.6-27B · vLLM · NVFP4 · conc 16
status: done
Configuration
| Model | Qwen/Qwen3.6-27B |
|---|---|
| Company | Alibaba |
| Family | Qwen |
| Parameters | 27B (dense) |
| Engine | vLLM |
| Quant / precision | NVFP4 |
| Why this quant | conc-16 base (non-spec) point of the Qwen3.6-27B NVFP4 sweep — matched no-spec baseline for the MTP conc-16 row (EXPERIMENTS.md |
| Download | unsloth/Qwen3.6-27B-NVFP4 |
| Context window | 65536 |
| Input modalities | text, image, video (served text-only here) |
Measured results
| Prefill tok/s | 147.97 |
|---|---|
| Decode tok/s | 116.57 |
| Peak memory (GB) | 107.72 (system MemAvailable delta (10s sampling) — vLLM static KV reservation (util 0.85)) |
| Completed | 2026-07-01 22:58 +0800 |
Full run command
# conc-16 base (no spec). Same recipe as the published conc-32 base — only --max-num-seqs differs.
scripts/bench-vllm-serving.sh unsloth/Qwen3.6-27B-NVFP4 65536 16 1000 600 256 \
--trust-remote-code --dtype bfloat16
Notes
conc-16 base (no-spec) point of the Qwen3.6-27B NVFP4 sweep — completes the base curve at c1/2/4/8/16/32 (matched baseline for the MTP conc-16 row, EXPERIMENTS.md #4/#14).
- Result (conc 16): prefill 147.97 / decode 116.57 tok/s aggregate; 0 errors; peak mem 107.7 GB.
- The full base decode curve: 9.33 → 18.11 → 35.54 → 67.06 → 116.57 → 187.74 tok/s at c1/2/4/8/16/32. Scaling factor per doubling falls from ~1.94 (c1→2) to 1.74 (c8→16) to 1.61 (c16→32) — the dense 27B transitions from bandwidth-bound (near-linear) toward compute-bound (diminishing returns) as the batch fills.
- MTP-vs-base delta on the
-mtp-c16page — this is where spec-decode’s edge is expected to compress, since a filling dense batch leaves less spare compute for the draft. - Sweep siblings (base):
-c1·-c2·-c4·-c8·c32(main).