Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 4

status: done

Configuration

Model	`Qwen/Qwen3.6-35B-A3B`
Company	Alibaba
Family	Qwen
Parameters	35B / 3B (MoE)
Engine	vLLM
Quant / precision	NVFP4
Why this quant	conc-4 base (non-spec) point of the Qwen3.6-35B-A3B NVFP4 sweep — matched no-spec baseline for the MTP/DFlash conc-4 rows (EXPERIMENTS.md
Download	nvidia/Qwen3.6-35B-A3B-NVFP4
Context window	65536
Input modalities	text, image, video (served text-only here)

Measured results

Prefill tok/s	181.46
Decode tok/s	173.76
Peak memory (GB)	107.73 (system MemAvailable delta (10s sampling) — vLLM static KV reservation (util 0.85))
Completed	2026-07-01 14:24 +0800

Full run command

# conc-4 base (no spec). Same recipe as the published conc-32 base — only --max-num-seqs differs.
scripts/bench-vllm-serving.sh nvidia/Qwen3.6-35B-A3B-NVFP4 65536 4 1000 600 256 \
  --quantization modelopt --trust-remote-code --reasoning-parser qwen3
# 411/1000 prompts (hit 600 s cap), 0 errors. ready after 385 s.

qwen3.6-35b-a3b Alibaba Qwen NVFP4 16-40B Spark recipe conc-4

Notes

conc-4 base (no-spec) point of the Qwen3.6-35B-A3B NVFP4 sweep — matched baseline for the MTP/DFlash conc-4 rows (EXPERIMENTS.md #4/#14). Same NVIDIA ModelOpt NVFP4 recipe as the published conc-32 base; only --max-num-seqs changes.

Result (conc 4): prefill 181.46 / decode 173.76 tok/s aggregate; 411/1000 prompts (hit the 600 s cap), 0 errors; peak mem 107.7 GB.
MTP speedup at conc-4: MTP 232.4 vs this base 173.76 = +33.8%. Across the full sweep the MTP-vs-base ratio is non-monotone (+25.6/+42.4/+33.8/+19.5/+30.3/+25.7% at c1/2/4/8/16/32), which turns out to be an MTP-measurement-window artifact rather than a real curve shape — see the -c16 page. Net: MTP wins a robust ~+20–42% (centered ~+30%), and the single-stream +25.6% is not the maximum.
TPOT 0.0 = qwen3 reasoning-parser client artifact — decode tok/s is the reported metric.
Sweep siblings: -c1 · -c2 · -c8 · -c16 · c32 (main). MTP counterpart: -mtp-c4.