Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 16

status: done

Configuration

ModelQwen/Qwen3.6-35B-A3B
CompanyAlibaba
FamilyQwen
Parameters35B / 3B (MoE)
EnginevLLM
Quant / precisionNVFP4
Why this quantconc-16 base (non-spec) point of the Qwen3.6-35B-A3B NVFP4 sweep — matched no-spec baseline for the MTP/DFlash conc-16 rows (EXPERIMENTS.md
Downloadnvidia/Qwen3.6-35B-A3B-NVFP4
Context window65536
Input modalitiestext, image, video (served text-only here)

Measured results

Prefill tok/s349.96
Decode tok/s332.42
Peak memory (GB)108.83 (system MemAvailable delta (10s sampling) — vLLM static KV reservation (util 0.85))
Completed2026-07-01 14:58 +0800

Full run command

# conc-16 base (no spec). Same recipe as the published conc-32 base — only --max-num-seqs differs.
scripts/bench-vllm-serving.sh nvidia/Qwen3.6-35B-A3B-NVFP4 65536 16 1000 600 256 \
  --quantization modelopt --trust-remote-code --reasoning-parser qwen3
# 784/1000 prompts (hit 600 s cap), 0 errors. ready after ~385 s.

qwen3.6-35b-a3b Alibaba Qwen NVFP4 16-40B Spark recipe conc-16

Notes

conc-16 base (no-spec) point of the Qwen3.6-35B-A3B NVFP4 sweep — completes the matched base line for the money-chart concurrencies (EXPERIMENTS.md #4/#14). Same NVIDIA ModelOpt NVFP4 recipe as the published conc-32 base; only --max-num-seqs changes.

  • Result (conc 16): prefill 349.96 / decode 332.42 tok/s aggregate; 784/1000 prompts (hit the 600 s cap), 0 errors; peak mem 108.8 GB.
  • The base line is clean and monotone: decode 74.7 (c1) → 113.2 (c2) → 173.8 (c4) → 241.8 (c8) → 332.4 (c16) → 430.8 (c32). A textbook throughput-vs-concurrency curve, no anomalies.
  • MTP-vs-base ratio — a robust ~+25–30% across the sweep, with a modest low-batch bump. Ratios (MTP c1/c8 at matched 600 s cap): +32.5% (c1) / +42.4% (c2) / +33.8% (c4) / +25.7% (c8) / +30.3% (c16) / +25.7% (c32). The apparent c8 “dip” (+19.5%) in the first pass was a short-cap artifact: the June MTP c8 was a 300 s-cap run (289 tok/s); a matched 600 s-cap recheck gave 304 tok/s → +25.7%, in line with the rest. So the curve is essentially flat ~+25–30% with a small c2–c4 peak (+42%/+34%) — a little batching still leaves spare compute for the 3-token draft — NOT a monotone decay. (Least-clean point is c2: base c2 was a 500-prompt run and MTP c2 a 900 s run, so the +42% there mixes caps.) For the fully single-session apples-to-apples decay story, the DFlash-vs-MTP money chart is cleaner than this base-vs-MTP ratio.
  • TPOT 0.0 = qwen3 reasoning-parser client artifact — decode tok/s is the reported metric.
  • Sweep siblings: -c1 · -c2 · -c4 · -c8 · c32 (main). MTP counterpart: -mtp-c16.