Qwen3.6-27B AEON Uncensored · vLLM · NVFP4 + MTP (XS) · conc 8
status: done
Configuration
| Model | AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored |
|---|---|
| Company | Alibaba |
| Family | Qwen |
| Parameters | 27B (dense) + grafted MTP head |
| Engine | vLLM + MTP (qwen3_5_mtp, grafted) (speculative decoding) |
| Quant / precision | NVFP4 (XS mixed-precision) |
| Why this quant | Concurrency-8 point of the AEON-7 NVFP4-MTP-XS sweep — same trusted native-MTP stack as the conc-32 run (stock vLLM, XS mixed-precision modelopt_fp4 + grafted Qwen3_5MTP head, no custom container). Lower-batch latency characterization; acceptance should hold ~constant vs conc-32 (workload-driven). |
| Download | AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-Multimodal-NVFP4-MTP-XS |
| Context window | 65536 |
| Input modalities | text, image, video (served text-only here) |
Measured results
| Prefill tok/s | 132.2 |
|---|---|
| Decode tok/s | 127.52 |
| Peak memory (GB) | 108.45 (system MemAvailable delta (10s sampling) — NVFP4 (XS mixed) + grafted MTP head) |
| Completed | 2026-06-23 14:11 +08 |
Full run command
# conc-8 latency point (500 prompts / 300 s cap). Trusted native-MTP path on STOCK vLLM (no custom
# container / DFlash). Same XS mixed-precision stack as the conc-32 run.
scripts/bench-vllm-serving.sh AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-Multimodal-NVFP4-MTP-XS \
65536 8 500 300 256 \
--trust-remote-code --quantization modelopt \
--speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":3}'
# 156/500 prompts (hit 300 s cap), 0 errors. ready after 422 s. TTFT median 505 ms, TPOT median 52.2 ms.
Notes
conc-8 point of the AEON-7 NVFP4-MTP-XS sweep. Lower-batch latency characterization (cap 500 prompts / 300 s — reached 156). Pairs with the conc-32 done run (decode 303 tok/s agg) and the conc-1 sibling. Trusted native-MTP path on stock vLLM — no custom container.
- Result (conc 8): prefill 132.2 / decode 127.52 tok/s aggregate; peak mem 108.45 GB; TTFT median 505 ms, TPOT median 52.2 ms (valid — no reasoning-parser). Lower aggregate than conc-32 (303) purely because 8 streams push fewer total tokens; the per-stream TPOT is the meaningful low-conc signal.
- Acceptance holds: mean accept-len ~3.0 (median 3.02), avg draft acceptance ~67.5% — identical to conc-32 (~3.0/~66%). Workload-driven, not concurrency-driven, same as the rest of the sweep.
- vs stock: decode 127.5 vs stock Qwen3.6-27B NVFP4+MTP conc-8 (109.05) = ~17% above — consistent with the conc-32 gap (~11%). Same caveat: different model + XS recipe, not a clean comparison.