Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 8
status: done
Configuration
| Model | Qwen/Qwen3.6-35B-A3B |
|---|---|
| Company | Alibaba |
| Family | Qwen |
| Parameters | 35B / 3B (MoE) |
| Engine | vLLM |
| Quant / precision | NVFP4 |
| Why this quant | conc-8 base (non-spec) point of the Qwen3.6-35B-A3B NVFP4 sweep — matched no-spec baseline for the MTP/DFlash conc-8 rows (EXPERIMENTS.md |
| Download | nvidia/Qwen3.6-35B-A3B-NVFP4 |
| Context window | 65536 |
| Input modalities | text, image, video (served text-only here) |
Measured results
| Prefill tok/s | 264.33 |
|---|---|
| Decode tok/s | 241.82 |
| Peak memory (GB) | 107.94 (system MemAvailable delta (10s sampling) — vLLM static KV reservation (util 0.85)) |
| Completed | 2026-07-01 14:39 +0800 |
Full run command
# conc-8 base (no spec). Same recipe as the published conc-32 base — only --max-num-seqs differs.
scripts/bench-vllm-serving.sh nvidia/Qwen3.6-35B-A3B-NVFP4 65536 8 1000 600 256 \
--quantization modelopt --trust-remote-code --reasoning-parser qwen3
# 574/1000 prompts (hit 600 s cap), 0 errors. ready after 385 s.
Notes
conc-8 base (no-spec) point of the Qwen3.6-35B-A3B NVFP4 sweep — matched baseline for the MTP/DFlash conc-8 rows (EXPERIMENTS.md #4/#14), so the post can quote the MTP/DFlash speedups at conc-8 against a real base instead of only conc-32. Same NVIDIA ModelOpt NVFP4 recipe as the published conc-32 base.
- Result (conc 8): prefill 264.33 / decode 241.82 tok/s aggregate; 574/1000 prompts (hit the 600 s cap), 0 errors; peak mem 107.9 GB.
- MTP speedup at conc-8: the original June MTP c8 (289.1, a 300 s-cap run) gave +19.5%, but a matched
600 s-cap recheck (2026-07-01) measured MTP c8 = 304.0, i.e. +25.7% over this base — the +19.5% “dip”
was a short-cap sampling artifact. With that correction the MTP-vs-base ratio is a robust ~+25–30%
across the sweep — +32.5% (c1) / +42.4% (c2) / +33.8% (c4) / +25.7% (c8) / +30.3% (c16) / +25.7% (c32)
— essentially flat with a modest low-batch (c2–c4) bump, NOT a monotone decay (full discussion on the
-c16page). DFlash at conc-8 (269.9) = +11.6% over base, positive but below MTP. - TPOT 0.0 =
qwen3reasoning-parser client artifact — decode tok/s is the reported metric. - Sweep siblings:
-c1·-c2·-c4·-c16·c32(main). MTP counterpart:-mtp-c8.