Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 1

status: done

Configuration

ModelQwen/Qwen3.6-35B-A3B
CompanyAlibaba
FamilyQwen
Parameters35B / 3B (MoE)
EnginevLLM
Quant / precisionNVFP4
Why this quantconc-1 base (non-spec) point of the Qwen3.6-35B-A3B NVFP4 sweep — the matched no-spec baseline for the MTP/DFlash conc-1 rows (EXPERIMENTS.md
Downloadnvidia/Qwen3.6-35B-A3B-NVFP4
Context window65536
Input modalitiestext, image, video (served text-only here)

Measured results

Prefill tok/s93.25
Decode tok/s74.74
Peak memory (GB)108.53 (system MemAvailable delta (10s sampling) — vLLM static KV reservation (util 0.85))
Completed2026-07-01 13:50 +0800

Full run command

# conc-1 base (no spec). Same recipe as the published conc-32 base — only --max-num-seqs differs.
scripts/bench-vllm-serving.sh nvidia/Qwen3.6-35B-A3B-NVFP4 65536 1 400 600 256 \
  --quantization modelopt --trust-remote-code --reasoning-parser qwen3
# 176/400 prompts (hit 600 s cap), 0 errors. ready after 362 s.

qwen3.6-35b-a3b Alibaba Qwen NVFP4 16-40B Spark recipe conc-1

Notes

conc-1 base (no-spec) point of the Qwen3.6-35B-A3B NVFP4 sweep — the matched single-stream baseline the post needed to quote MTP/DFlash speedups at conc-1 (EXPERIMENTS.md #4/#14). Same NVIDIA ModelOpt NVFP4 recipe as the published conc-32 base; only --max-num-seqs changes.

  • Result (conc 1): prefill 93.25 / decode 74.74 tok/s aggregate; 176/400 prompts (hit the 600 s cap), 0 errors; peak mem 108.5 GB.
  • MTP speedup at conc-1: against a matched 600 s-cap MTP recheck (99.04 tok/s) this base gives +32.5% (the published MTP c1 of 93.9 was a 300 s-cap run, ~5% under-measured; see -mtp-c1). DFlash at conc-1 (101.9) = +36.4% over base.
  • Consequence for the money chart: DFlash’s single-stream edge over MTP is only ~+2.9% (101.9 vs the matched MTP 99.04), not the +8.5% quoted against the short-cap MTP — and even that is ctx-confounded (DFlash ran ctx 40960 vs MTP 65536). So DFlash barely leads MTP at conc-1 and loses from conc-2 on. TPOT 0.0 = qwen3 reasoning-parser client artifact — decode tok/s is real.
  • Sweep siblings: -c2 · -c4 · -c8 · -c16 · c32 (main). MTP counterpart: -mtp-c1.