Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 16
status: done
Configuration
| Model | Qwen/Qwen3.6-35B-A3B |
|---|---|
| Company | Alibaba |
| Family | Qwen |
| Parameters | 35B / 3B (MoE) |
| Engine | vLLM |
| Quant / precision | NVFP4 |
| Why this quant | conc-16 base (non-spec) point of the Qwen3.6-35B-A3B NVFP4 sweep — matched no-spec baseline for the MTP/DFlash conc-16 rows (EXPERIMENTS.md |
| Download | nvidia/Qwen3.6-35B-A3B-NVFP4 |
| Context window | 65536 |
| Input modalities | text, image, video (served text-only here) |
Measured results
| Prefill tok/s | 349.96 |
|---|---|
| Decode tok/s | 332.42 |
| Peak memory (GB) | 108.83 (system MemAvailable delta (10s sampling) — vLLM static KV reservation (util 0.85)) |
| Completed | 2026-07-01 14:58 +0800 |
Full run command
# conc-16 base (no spec). Same recipe as the published conc-32 base — only --max-num-seqs differs.
scripts/bench-vllm-serving.sh nvidia/Qwen3.6-35B-A3B-NVFP4 65536 16 1000 600 256 \
--quantization modelopt --trust-remote-code --reasoning-parser qwen3
# 784/1000 prompts (hit 600 s cap), 0 errors. ready after ~385 s.
Notes
conc-16 base (no-spec) point of the Qwen3.6-35B-A3B NVFP4 sweep — completes the matched base line for
the money-chart concurrencies (EXPERIMENTS.md #4/#14). Same NVIDIA ModelOpt NVFP4 recipe as the published
conc-32 base; only --max-num-seqs changes.
- Result (conc 16): prefill 349.96 / decode 332.42 tok/s aggregate; 784/1000 prompts (hit the 600 s cap), 0 errors; peak mem 108.8 GB.
- The base line is clean and monotone: decode 74.7 (c1) → 113.2 (c2) → 173.8 (c4) → 241.8 (c8) → 332.4 (c16) → 430.8 (c32). A textbook throughput-vs-concurrency curve, no anomalies.
- MTP-vs-base ratio — a robust ~+25–30% across the sweep, with a modest low-batch bump. Ratios (MTP c1/c8 at matched 600 s cap): +32.5% (c1) / +42.4% (c2) / +33.8% (c4) / +25.7% (c8) / +30.3% (c16) / +25.7% (c32). The apparent c8 “dip” (+19.5%) in the first pass was a short-cap artifact: the June MTP c8 was a 300 s-cap run (289 tok/s); a matched 600 s-cap recheck gave 304 tok/s → +25.7%, in line with the rest. So the curve is essentially flat ~+25–30% with a small c2–c4 peak (+42%/+34%) — a little batching still leaves spare compute for the 3-token draft — NOT a monotone decay. (Least-clean point is c2: base c2 was a 500-prompt run and MTP c2 a 900 s run, so the +42% there mixes caps.) For the fully single-session apples-to-apples decay story, the DFlash-vs-MTP money chart is cleaner than this base-vs-MTP ratio.
- TPOT 0.0 =
qwen3reasoning-parser client artifact — decode tok/s is the reported metric. - Sweep siblings:
-c1·-c2·-c4·-c8·c32(main). MTP counterpart:-mtp-c16.