gpt-oss-20b · vLLM · MXFP4 + EAGLE3 · conc 4
status: done
Configuration
| Model | openai/gpt-oss-20b |
|---|---|
| Company | OpenAI |
| Family | gpt-oss |
| Parameters | 21B / 3.6B (MoE) |
| Engine | vLLM + EAGLE3 (speculative decoding) |
| Quant / precision | MXFP4 |
| Why this quant | conc-4 point of the EAGLE3 fine-grained sweep (EXPERIMENTS.md |
| Download | openai/gpt-oss-20b |
| Context window | 65536 |
| Input modalities | text |
Measured results
| Prefill tok/s | 127.09 |
|---|---|
| Decode tok/s | 85.36 |
| Peak memory (GB) | 109.15 (system MemAvailable delta (10s sampling) — vLLM static KV reservation (util 0.85) + EAGLE3 head) |
| Completed | 2026-07-01 16:15 +0800 |
Full run command
# conc-4 EAGLE3, cu130-nightly + RedHatAI speculator. Harmony vocab via VOCAB_DIR override.
VLLM_IMAGE=vllm/vllm-openai:cu130-nightly VOCAB_DIR=$HOME/tiktoken_encodings \
scripts/bench-vllm-serving.sh openai/gpt-oss-20b 65536 4 1000 600 256 \
--speculative-config '{"model":"RedHatAI/gpt-oss-20b-speculator.eagle3","method":"eagle3","num_speculative_tokens":3}'
# 212/1000 prompts (hit 600 s cap), 15 harmony errors. ready after 162 s.
Notes
conc-4 EAGLE3 — still a net loss, deepening. gpt-oss-20b MXFP4 + RedHatAI EAGLE3 speculator on vLLM, conc 4.
- Result (conc 4): prefill 127.09 / decode 85.36 tok/s aggregate; 212/1000 prompts (hit the 600 s cap), 15 harmony errors; peak mem 109.2 GB.
- EAGLE3 vs base at conc-4: −32.9% (85.36 vs base 127.26) — worse than the conc-2 loss (−29.3%). The spec penalty deepens as the batch fills, exactly the wrong direction for a real draft win. Running total for #15: −29.3% (c2) → −32.9% (c4), both strongly negative.
- Acceptance dismal: mean accept-len ~1.15, avg draft acceptance ~5% — the draft converts almost nothing on this ShareGPT+harmony workload, so it’s pure overhead.
- TTFT/TPOT are buffered-reasoning artifacts — aggregate decode tok/s is the valid metric.
- Sweep:
-c1·-c2·-c8·-c16·c32(main). Base:-c4.