Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 32

status: done

Configuration

Model	`nvidia/Qwen3.6-35B-A3B-NVFP4`
Company	Alibaba
Family	Qwen
Parameters	35B / 3B (MoE, hybrid GDN+full-attn) + DFlash external drafter
Engine	vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) (speculative decoding)
Quant / precision	NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates)
Why this quant	conc-32 point of the ctx-matched (65536) DFlash money-chart sweep — the last, highest-batch endpoint (2026-07-02). Completes the six-point line matched exactly to base/MTP context. Same one-boot protocol (official checkpoint, small-page drafter, max-num-seqs 64). SAFETY — untrusted third-party image; NO credentials, weights + drafter READ-ONLY, port loopback-only.
Download	nvidia/Qwen3.6-35B-A3B-NVFP4
Context window	65536
Input modalities	text, image, video (served text-only here)

Measured results

Prefill tok/s	406.34
Decode tok/s	407.07
Peak memory (GB)	111.3 (system MemAvailable delta (idle baseline 118 GiB → 6.7 GiB available at load) over the one-boot conc-1/2/4/8/16/32 sweep — vLLM static KV (util 0.85) + DFlash drafter)
Completed	2026-07-02 07:21 +0800

Full run command

# UNTRUSTED image — NO creds; weights + drafter READ-ONLY; loopback port. ONE boot (ctx 65536,
# max-num-seqs 64, small-page drafter @31977fbe) sweeping client conc 1/2/4/8/16/32 — see the main
# (c1) page for the full docker run command. Only the client --concurrency differs.
python3 scripts/bench-serving.py --base-url http://127.0.0.1:8000 --model official \
  --dataset benchmark_data/ShareGPT_V3_unfiltered_cleaned_split.json --concurrency 32 --num-prompts 1000 --max-seconds 600 --max-tokens 256
# 979/1000 prompts (hit 600 s cap), 0 errors.

qwen3.6-35b-a3b Alibaba Qwen NVFP4 16-40B Spark recipe conc-32

Notes

conc-32 point of the Qwen3.6-35B-A3B NVFP4 + DFlash line — NEW page, completes the ctx-matched (65536) six-point sweep. Official nvidia/Qwen3.6-35B-A3B-NVFP4 on the AEON image, DFlash n=11 via the small-page drafter, one-boot sweep. 0 errors.

Result (conc 32): prefill 406.34 / decode 407.07 tok/s aggregate; 979/1000 prompts (nearly full coverage, 600 s cap), 0 errors. vs MTP conc-32 (541.26): −24.8%.
The DFlash money chart is now a fully-sampled, fully ctx-matched curve. Six points, one boot, one context (65536), one client protocol: decode 99.76 (c1) → 151.3 (c2) → 210.7 (c4) → 267.4 (c8) → 340.2 (c16) → 407.1 (c32); vs MTP: +0.7% → −6.1% → −9.3% → −12.0% → −21.5% → −24.8%. This is within 1.5% of the old ctx-40960 conc-32 reading (401.2, −25.9% vs the same MTP baseline) — at this batch size the extra 24576 tokens of context barely move the needle; the compute-bound collapse is a genuine property of the draft, not a context artifact. The only real correction from the ctx fix is at the low end: DFlash’s conc-1 “win” was entirely a context-length artifact — matched, it’s a wash.
Why — wasted draft compute. DFlash drafts 11 at ~26.1% acceptance / accept-len ~3.87 vs MTP’s 3 at ~67% / 3.0 — ≈7 wasted drafter forward passes per step. At conc-32 those wasted passes compete directly with real decode work for every available GPU cycle, and DFlash falls furthest behind MTP here. Acceptance stays flat vs conc across the whole sweep (~26–27% throughout, workload-driven) — the collapse is pure draft-efficiency economics, not an acceptance drop under load.
Bottom line, footnote removed: the money chart’s DFlash line is now complete and matched to base/MTP at every point (context 65536, one client protocol, one boot). Native MTP wins at every concurrency — DFlash ties at best (c1) and loses increasingly from c2 onward. Keep MTP for any mixed-workload gateway on this hybrid-GDN 35B-A3B target.
One server lifetime for the whole six-point sweep → mem is the single ~111.3 GB reservation. TPOT 0.0 = qwen3 reasoning-parser client artifact.
Series: c1 (main) · c2 · c4 · c8 · c16. Matched MTP: -mtp (the no-suffix MTP page is conc-32).