Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-1

status: done

Configuration

Model	`Qwen/Qwen3.8-27B`
Company	Alibaba
Family	Qwen
Parameters	27B (dense, hybrid linear-attn)
Engine	SGLang + DFlash2 (speculative decoding)
Quant / precision	NVFP4
Why this quant	Same Spark-recipe NVFP4 target (RadixArk/Qwen3.8-27B-NVFP4) as the DSpark line, but DFlash2 block-diffusion draft (z-lab/Qwen3.8-27B-DFlash2 @50307d4) on the derived lmsysorg/sglang:qwen38-27b-dflash2 image. mem-fraction-static 0.90. conc-1 point of the ShareGPT sweep (client conc 1/2/4/8/16/32).
Download	RadixArk/Qwen3.8-27B-NVFP4
Context window	262144
Input modalities	text, image, video (served text-only here)

Measured results

Prefill tok/s	44.43
Decode tok/s	30.04
Peak memory (GB)	109.69 (system MemAvailable delta (10s sampling) over the DFlash2 boot (MAX_CONCURRENT_REQUESTS=10))
Completed	2026-08-20 05:04 +0800

Full run command

# MiaAI-Lab Spark recipe, native 262K, YaRN off, mem-fraction-static 0.90.
cd ~/Desktop/Qwen3.8-27B-SGLang-DGX-Spark
MAX_CONCURRENT_REQUESTS=10 ./start-dflash.sh
python3 ~/Desktop/autobench/scripts/bench-serving.py \
  --base-url http://127.0.0.1:8888 --model qwen3.8-27b-sglang \
  --dataset ~/Desktop/autobench/benchmark_data/ShareGPT_V3_unfiltered_cleaned_split.json \
  --num-prompts 1000 --max-seconds 900 --concurrency 1 --max-tokens 256
# RESULT prefill_toks=44.43 decode_toks=30.04 completed=116 errors=0 duration_s=906.2 hit_time_cap=True
# image lmsysorg/sglang:qwen38-27b-dflash2@sha256:8a1a8f540fae7b1b9d1e8ee72ff279dc5ae3a9a3e4f83172f659b11d76f6f175

qwen3.8-27b Alibaba Qwen NVFP4 16-40B Spark recipe conc-1

Notes

conc-1 point of the Qwen3.8-27B SGLang NVFP4 + DFlash2 Spark-recipe line. MiaAI-Lab start-dflash.sh on derived lmsysorg/sglang:qwen38-27b-dflash2 (upstream DFLASH2 overlay + in-place quantized-head patch — no dense lm_head dequant; that path hard-rebooted this box). Draft z-lab/Qwen3.8-27B-DFlash2@50307d4, 8 draft tokens, --mem-fraction-static 0.90, native 262K, YaRN off. This conc ran with MAX_CONCURRENT_REQUESTS=10 (DFLASH uses 5 mamba slots/req so max_running_requests is floor(MAX*4/5)). CUDA-graph decode cap was pinned to the running-request ceiling.

Result (conc 1): prefill 44.43 / decode 30.04 tok/s aggregate; 116/1000 prompts, 0 errors, duration 906.2s (hit the 900 s time cap). TTFT median 220.4 ms, TPOT median 31.5 ms, req throughput 0.128/s.
vs DSpark (same checkpoint, same ShareGPT): DFlash2 c1 30.0 vs DSpark 23.4 (+28%); c2 56.2 vs 43.4 (+29%). c4 85.1 vs 75.5 is a 69 s / 27-request snippet after the engine died — do not splice it onto the DSpark c4 15-min point as a clean race.
vs vLLM MTP (unsloth NVFP4, ctx 65536): MTP c1 21.35. DFlash2 c1 30.0 is the single-stream winner on this box; counting is completion_tokens (this DFlash2 image batches several tokens per SSE event — event-counting would under-read ~4×, as MiaAI documented).
Acceptance: ~3.2–4.0 tokens/step of 8 (~32–43%). Better than DSpark’s ~25% on ShareGPT; still far from 70–85% MTP/EAGLE3. Workload-driven, not a draft/target mismatch (official z-lab DFlash2 against RadixArk NVFP4, quantized-head selector in-place).
Memory / crashes: peak 109.69 GB. First-request GC freeze (Freezing GC in Scheduler process) plus 0.90 reservation sits at ~7–9 GB free; the 6 GB watchdog SIGKILLs (exit 137). Recipe 0.95 hard-rebooted this box at draft-graph capture before the quantized-head patch; we never went above 0.90. DFLASH extra_buffer (not lazy) + 5 state slots/req.
Series: c1 · c2 · c4 · c8 · c16 · c32. Matched DSpark: c1 · c2 · c4 · c8 · c16 · c32.