Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 2
status: done
Configuration
| Model | Qwen/Qwen3.6-35B-A3B |
|---|---|
| Company | Alibaba |
| Family | Qwen |
| Parameters | 35B / 3B (MoE) |
| Engine | vLLM |
| Quant / precision | NVFP4 |
| Why this quant | conc-2 base (non-spec) point of the Qwen3.6-35B-A3B NVFP4 sweep — matched no-spec baseline for the MTP/DFlash conc-2 rows (EXPERIMENTS.md |
| Download | nvidia/Qwen3.6-35B-A3B-NVFP4 |
| Context window | 65536 |
| Input modalities | text, image, video (served text-only here) |
Measured results
| Prefill tok/s | 147.36 |
|---|---|
| Decode tok/s | 113.19 |
| Peak memory (GB) | 107.8 (system MemAvailable delta (10s sampling) — vLLM static KV reservation (util 0.85)) |
| Completed | 2026-07-01 14:07 +0800 |
Full run command
# conc-2 base (no spec). Same recipe as the published conc-32 base — only --max-num-seqs differs.
scripts/bench-vllm-serving.sh nvidia/Qwen3.6-35B-A3B-NVFP4 65536 2 500 600 256 \
--quantization modelopt --trust-remote-code --reasoning-parser qwen3
# 267/500 prompts (hit 600 s cap), 0 errors. ready after 404 s.
Notes
conc-2 base (no-spec) point of the Qwen3.6-35B-A3B NVFP4 sweep — matched baseline for the MTP/DFlash
conc-2 rows (EXPERIMENTS.md #4/#14). Same NVIDIA ModelOpt NVFP4 recipe as the published conc-32 base; only
--max-num-seqs changes.
- Result (conc 2): prefill 147.36 / decode 113.19 tok/s aggregate; 267/500 prompts (hit the 600 s cap), 0 errors; peak mem 107.8 GB.
- MTP speedup at conc-2: MTP 161.2 vs this base 113.19 = +42.4% — larger than the conc-1 speedup (+25.6%). The base-vs-MTP win is not monotone-decaying from conc-1; it rises into low-batch before decaying at high concurrency (see the c4/c8/c16 siblings for the full curve). Worth flagging against the post’s “MTP win shrinks as the batch fills” framing — the shrink is real at the high end, but there’s a low-batch peak first.
- TPOT 0.0 =
qwen3reasoning-parser client artifact — decode tok/s is the reported metric. - Sweep siblings:
-c1·-c4·-c8·-c16·c32(main). MTP counterpart:-mtp-c2.