gpt-oss-20b · vLLM · MXFP4 · conc 16
status: done
Configuration
| Model | openai/gpt-oss-20b |
|---|---|
| Company | OpenAI |
| Family | gpt-oss |
| Parameters | 21B / 3.6B (MoE) |
| Engine | vLLM |
| Quant / precision | MXFP4 |
| Why this quant | conc-16 base (non-spec) point of gpt-oss-20b — matched no-spec baseline for the EAGLE3 conc-16 row (EXPERIMENTS.md |
| Download | openai/gpt-oss-20b |
| Context window | 65536 |
| Input modalities | text |
Measured results
| Prefill tok/s | 403.31 |
|---|---|
| Decode tok/s | 340.69 |
| Peak memory (GB) | 107.83 (system MemAvailable delta (10s sampling) — vLLM static KV reservation (util 0.85)) |
| Completed | 2026-07-01 15:35 +0800 |
Full run command
# conc-16 base (no spec), cu130-nightly. Harmony vocab via VOCAB_DIR override.
VLLM_IMAGE=vllm/vllm-openai:cu130-nightly VOCAB_DIR=$HOME/tiktoken_encodings \
scripts/bench-vllm-serving.sh openai/gpt-oss-20b 65536 16 1000 600 256
# 837/1000 prompts (hit 600 s cap), 89 harmony errors.
Notes
conc-16 base (no-spec) point of gpt-oss-20b MXFP4 — completes the matched base line for the EAGLE3 artifact test (EXPERIMENTS.md #15). Same cu130-nightly recipe as the published conc-32 base.
- Result (conc 16): prefill 403.31 / decode 340.69 tok/s aggregate; 837/1000 prompts (hit the 600 s cap), 89 harmony errors (error count scales cleanly with batch: 5→22→89 at c2→c4→c16, all mid-reasoning 256-tok truncation — a vLLM harmony-finalizer robustness cost, not a config fault); peak mem 107.8 GB.
- Base line for the artifact test is now: decode 83.4 (c2) → 127.3 (c4) → 340.7 (c16) → 535.3 (c32). Pair each against the EAGLE3 row to see whether spec-decode helps at all before conc-32.
- TTFT/TPOT are buffered-reasoning artifacts — aggregate decode tok/s is the valid metric.
- Base siblings:
-c2·-c4·c32(main). EAGLE3 counterpart:-eagle3-c16.