Qwen3.6-27B AEON Uncensored · vLLM · NVFP4 + MTP (XS) · conc 1
status: done
Configuration
| Model | AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored |
|---|---|
| Company | Alibaba |
| Family | Qwen |
| Parameters | 27B (dense) + grafted MTP head |
| Engine | vLLM + MTP (qwen3_5_mtp, grafted) (speculative decoding) |
| Quant / precision | NVFP4 (XS mixed-precision) |
| Why this quant | Single-stream point of the AEON-7 NVFP4-MTP-XS sweep — same trusted native-MTP stack as the conc-32 run (stock vLLM, XS mixed-precision modelopt_fp4 + grafted Qwen3_5MTP head, no custom container). Best-case MTP latency with no batch contention; the per-stream TPOT is the headline number. |
| Download | AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-Multimodal-NVFP4-MTP-XS |
| Context window | 65536 |
| Input modalities | text, image, video (served text-only here) |
Measured results
| Prefill tok/s | 3.74 |
|---|---|
| Decode tok/s | 19.59 |
| Peak memory (GB) | 109.28 (system MemAvailable delta (10s sampling) — NVFP4 (XS mixed) + grafted MTP head) |
| Completed | 2026-06-23 14:24 +08 |
Full run command
# conc-1 single-stream latency point (200 prompts / 300 s cap). Trusted native-MTP path on STOCK vLLM
# (no custom container / DFlash). Same XS mixed-precision stack as the conc-32 run.
scripts/bench-vllm-serving.sh AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-Multimodal-NVFP4-MTP-XS \
65536 1 200 300 256 \
--trust-remote-code --quantization modelopt \
--speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":3}'
# 23/200 prompts (single stream + cold load hit the 300 s cap — expected; per-stream latency is the point).
# 0 errors. ready after 413 s. TTFT median 281 ms, TPOT median 44.7 ms.
Notes
conc-1 (single-stream) point of the AEON-7 NVFP4-MTP-XS sweep. Best-case MTP latency — no batch contention. Pairs with the conc-32 done run (decode 303 tok/s agg) and the conc-8 sibling. Trusted native-MTP path on stock vLLM — no custom container.
- Result (conc 1): prefill 3.74 / decode 19.59 tok/s single-stream aggregate; 23/200 prompts (hit 300 s cap — single stream + cold load, expected); peak mem 109.28 GB; TTFT median 281 ms, TPOT median 44.7 ms — the headline low-conc number. TPOT is lower than conc-8 (52.2 ms): with one stream there’s no batch contention, so each token decodes faster.
- Acceptance holds: mean accept-len ~3.0 (median 3.03), avg draft acceptance ~67.7%, accepted throughput ~14.7 tok/s — identical to conc-8/conc-32. Acceptance is workload-driven, not concurrency-driven across the full AEON sweep (conc-1/8/32), exactly as on the stock Qwen3.6 sweeps.
- vs stock: single-stream decode 19.6 vs stock Qwen3.6-27B NVFP4+MTP conc-1 (16.85) — again modestly above, consistent with conc-8/conc-32. Same caveat: different model + XS recipe, not a clean comparison.