DGX Spark Autobench

Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP · conc 4

status: done

Configuration

ModelQwen/Qwen3.6-35B-A3B
CompanyAlibaba
FamilyQwen
Parameters35B / 3B (MoE)
EnginevLLM + MTP (speculative decoding)
Quant / precisionNVFP4
Why this quantconc-4 fine-grained point of the Qwen3.6-35B-A3B NVFP4 + native-MTP sweep — same stack as conc-1/8/32 (nvidia ModelOpt NVFP4 base on marlin + in-repo MTP head on triton). With conc-2 it pins the 1→8 crossover region for the money chart (EXPERIMENTS.md
Downloadnvidia/Qwen3.6-35B-A3B-NVFP4
Context window65536
Input modalitiestext, image, video (served text-only here)

Measured results

Prefill tok/s218.82
Decode tok/s232.43
Peak memory (GB)109.13 (system MemAvailable delta (10s sampling) — vLLM static KV (util 0.85) + MTP head)
Completed2026-07-01 12:37 +0800

Full run command

# conc-4 fine-grained point. Same NVIDIA DGX Spark MTP recipe as conc-1/8/32 — only --max-num-seqs differs.
scripts/bench-vllm-serving.sh nvidia/Qwen3.6-35B-A3B-NVFP4 65536 4 1000 600 256 \
  --quantization modelopt --trust-remote-code --reasoning-parser qwen3 --moe-backend marlin \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3,"moe_backend":"triton"}'
# 548/1000 prompts (hit 600 s time cap), 0 errors. ready after 379 s.

qwen3.6-35b-a3b Alibaba Qwen NVFP4 16-40B Spark recipe conc-4

Notes

conc-4 point of the Qwen3.6-35B-A3B NVFP4 + MTP sweep — fine-grained resolution for the money chart’s 1→8 crossover region (EXPERIMENTS.md #13/#14). Same NVIDIA ModelOpt NVFP4 base (marlin) + in-repo MTP head (triton) as the published conc-1/8/32 rows; only --max-num-seqs changes.

  • Result (conc 4): prefill 218.82 / decode 232.43 tok/s aggregate; 548/1000 prompts (hit the 600 s time cap), 0 errors; peak mem 109.1 GB. Sits between conc-2 (161.2) and conc-8 (289.1) — the MTP decode curve stays monotonic through the low-batch region.
  • Acceptance holds ~constant: mean accept-len ~3.0 (2.99–3.17), avg draft acceptance ~66–69%, per-position 0.84 / 0.66 / 0.52 — indistinguishable from the other conc points. Workload-driven, not concurrency-driven, as expected for the in-repo MTP head.
  • TPOT median reads 0.0 — the usual qwen3 reasoning-parser client artifact; trust the aggregate decode tok/s and the in-engine SpecDecoding throughput.
  • Sweep siblings: -c1 · -c2 · -c8 · -c16 · maxctx/c32.