Browse by concurrency
Configurations grouped by the number of parallel serves (--parallel / --max-num-seqs /
client concurrency) used for the run. The whole benchmark is run at a fixed concurrency for
cross-config comparability — this page makes that explicit and groups any future low-concurrency
(e.g. conc-1) variants separately. ← all tag kinds
conc-1conc-2conc-4conc-8conc-16conc-32conc-64conc-128conc-256
conc-1 (38)
| Configuration | Engine | Quant | Ctx | Conc | Decode tok/s | Status | Completed |
|---|---|---|---|---|---|---|---|
| Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-1 | SGLang + DFlash2 | NVFP4 | 262144 | 1 | 30.0 | done | 2026-08-20 05:04 +0800 |
| Qwen3.8-27B · SGLang · NVFP4 + DSpark · conc-1 | SGLang + DSpark | NVFP4 | 262144 | 1 | 23.4 | done | 2026-08-20 03:31 +0800 |
| Qwen3.8-27B · vLLM · NVFP4 + MTP · conc-1 | vLLM + MTP | NVFP4 | 65536 | 1 | 21.4 | done | 2026-08-15 11:56 +0800 |
| Qwen3.8-27B · vLLM · NVFP4 · conc-1 | vLLM | NVFP4 | 65536 | 1 | 11.2 | done | 2026-08-15 11:56 +0800 |
| Nemotron-3 Puzzle 75B-A9B · vLLM · NVFP4 · conc 1 | vLLM | NVFP4 | 65536 | 1 | 19.8 | done | 2026-07-11 16:29 +08 |
| DiffusionGemma-26B-A4B · vLLM · NVFP4 · conc 1 | vLLM | NVFP4 | 65536 | 1 | 116.0 | done | 2026-07-04 00:33 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 · conc 1 | vLLM + EAGLE3 | NVFP4 | 65536 | 1 | 50.1 | done | 2026-07-03 17:54 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + MTP · conc 1 | vLLM + MTP (Google assistant drafter) | NVFP4 | 65536 | 1 | 57.2 | done | 2026-07-03 16:32 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 · conc 1 | vLLM | NVFP4 | 65536 | 1 | 29.5 | done | 2026-07-03 15:15 +0800 |
| Gemma 4 31B · vLLM · NVFP4 + MTP · conc 1 | vLLM + MTP (Google assistant drafter) | NVFP4 | 65536 | 1 | 18.3 | done | 2026-07-02 15:52 +0800 |
| gpt-oss-20b · vLLM · MXFP4 · conc 1 | vLLM | MXFP4 | 65536 | 1 | 45.6 | done | 2026-07-02 07:51 +0800 |
| Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 1 | vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) | NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates) | 65536 | 1 | 99.8 | done | 2026-07-02 07:21 +0800 |
| Qwen3.6-27B · vLLM · NVFP4 · conc 1 | vLLM | NVFP4 | 65536 | 1 | 9.3 | done | 2026-07-01 21:44 +0800 |
| Qwen3-Coder-30B-A3B · DDTree vs DFlash · HumanEval (coding workload) · single-stream | DDTree research harness (github.com/liranringel/ddtree, PyTorch + transformers, batch-1) + DDTree tree draft (budgets 64/256) vs single-line DFlash — z-lab/Qwen3-Coder-30B-A3B-DFlash, block_size 16 | BF16 (harness loads the unquantized target via AutoModelForCausalLM) | 4096 | 1 | 49.3 | done | 2026-07-01 21:26 +0800 |
| Qwen3-Coder-30B-A3B · DDTree vs DFlash vs autoregressive · single-stream (batch-1) | DDTree research harness (github.com/liranringel/ddtree, PyTorch + transformers, batch-1) + DDTree tree draft (budgets 64/256) vs single-line DFlash — z-lab/Qwen3-Coder-30B-A3B-DFlash, block_size 16 | BF16 (harness loads the unquantized target via AutoModelForCausalLM) | 4096 | 1 | 20.8 | done | 2026-07-01 21:13 +0800 |
| gpt-oss-120b · vLLM · MXFP4 + EAGLE3 (LMSYS draft) · conc 1 | vLLM + EAGLE3 (lmsys/EAGLE3-gpt-oss-120b-bf16 — the SGLang/SpecForge draft, on vLLM) | MXFP4 | 65536 | 1 | 25.5 | done | 2026-07-01 17:37 +0800 |
| Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 1 | vLLM | NVFP4 | 65536 | 1 | 74.7 | done | 2026-07-01 13:50 +0800 |
| Qwen3.6-35B-A3B · DDTree (Diffusion Draft Tree) · BLOCKED (hybrid GDN cache) | DDTree research harness (github.com/liranringel/ddtree, PyTorch + transformers, batch-1) + DDTree — tree draft over the z-lab/Qwen3.6-35B-A3B-DFlash block-diffusion drafter | BF16 (harness loads the unquantized target via AutoModelForCausalLM) | 4096 | 1 | — | blocked | 2026-07-01 |
| Qwen3.6-27B (dense) · DDTree (Diffusion Draft Tree) · BLOCKED (hybrid GDN cache) | DDTree research harness (github.com/liranringel/ddtree, PyTorch + transformers, batch-1) + DDTree — tree draft over the z-lab/Qwen3.6-27B-DFlash block-diffusion drafter | BF16 (harness loads the unquantized target via AutoModelForCausalLM) | 4096 | 1 | — | blocked | 2026-07-01 |
| Qwen3.6-35B-A3B-heretic (AEON) · vLLM-ultimate · NVFP4 + DFlash · conc 1/8/32 | vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) | NVFP4 (compressed-tensors, nvfp4-pack-quantized — GDN gates kept in BF16) | 40960 | 1 | 77.6 | done | 2026-06-28 20:52 +0800 |
| Ornith-1.0-35B AEON Uncensored · vLLM (aeon-ultimate) · NVFP4 + DFlash · conc 1 — BLOCKED | vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash, num_speculative_tokens 6) — BLOCKED at KV unification | NVFP4 (compressed-tensors, nvfp4-pack-quantized) | 262144 | 1 | — | blocked | 2026-06-28 16:13 +0800 |
| Ornith-1.0-35B AEON Uncensored · vLLM (aeon-ultimate) · NVFP4 · conc 1 (256K ctx) | vLLM (aeon-vllm-ultimate custom container) + none (DFlash OFF — DFlash trips hybrid+draft KV unification, see dflash-blocked page) | NVFP4 (compressed-tensors, nvfp4-pack-quantized) | 262144 | 1 | 37.7 | done | 2026-06-28 16:13 +0800 |
| DeepSeek V4-Flash REAP-K180 · ds4 · NVFP4-hybrid · conc 1 (1M ctx) | ds4 (ds4-nvfp4-spark, custom GB10 runtime) + none — ds4 is a serial single-worker; the DeepSeek MTP head is not exercised by this runtime | NVFP4-hybrid (NVFP4 experts + Q2_K + Q8_0 + F16) | 1048576 | 1 | 11.4 | done | 2026-06-28 13:34 +08 |
| MiniMax-M2.7-REAP 172B · vLLM · NVFP4 (W4A4) · conc 1 · 160K | vLLM | NVFP4 (W4A4) | 163840 | 1 | 25.4 | done | 2026-06-27 19:22 +0800 |
| MiniMax-M2.5-REAP 139B · vLLM · NVFP4 (W4A16) · conc 1 · 192K | vLLM | NVFP4 (W4A16) | 196608 | 1 | 27.0 | done | 2026-06-27 15:22 +0800 |
| Qwen3.6-27B AEON Uncensored · vLLM-ultimate (custom) · NVFP4 + DFlash · conc 1 | vLLM (aeon-vllm-ultimate custom container) + DFlash (z-lab drafter, num_speculative_tokens 12) | NVFP4 (XS mixed-precision) | 65536 | 1 | 29.9 | done | 2026-06-24 00:47 +0800 |
| gpt-oss-120b · SGLang · MXFP4 + EAGLE3 · conc 1 | SGLang + EAGLE3 | MXFP4 | 65536 | 1 | 40.6 | done | 2026-06-23 20:08 +0800 |
| Qwen3.6-27B AEON Uncensored · vLLM · NVFP4 + MTP (XS) · conc 1 | vLLM + MTP (qwen3_5_mtp, grafted) | NVFP4 (XS mixed-precision) | 65536 | 1 | 19.6 | done | 2026-06-23 14:24 +08 |
| Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP · conc 1 | vLLM + MTP | NVFP4 | 65536 | 1 | 93.9 | done | 2026-06-23 13:02 +08 |
| Qwen3.6-27B · vLLM · NVFP4 + MTP · conc 1 | vLLM + MTP | NVFP4 | 65536 | 1 | 16.9 | done | 2026-06-23 10:48 +08 |
| Gemma 4 31B · vLLM · NVFP4 + EAGLE3 · conc 1 | vLLM + EAGLE3 | NVFP4 | 65536 | 1 | 14.8 | done | 2026-06-23 01:01 +08 |
| Gemma 4 31B · llama.cpp · Q4_K_M + MTP · conc 1 | llama.cpp + MTP (Google assistant drafter) | Q4_K_M | 65536 | 1 | 24.0 | done | 2026-06-23 00:18 +08 |
| Gemma 4 12B · llama.cpp · Q4_K_M + MTP · conc 1 | llama.cpp + MTP (Google assistant drafter) | Q4_K_M | 65536 | 1 | 49.2 | done | 2026-06-23 00:06 +08 |
| gpt-oss-120b · vLLM · MXFP4 + EAGLE3 · conc 1 | vLLM + EAGLE3 | MXFP4 | 65536 | 1 | 14.7 | done | 2026-06-22 23:29 +08 |
| gpt-oss-20b · vLLM · MXFP4 + EAGLE3 · conc 1 | vLLM + EAGLE3 | MXFP4 | 65536 | 1 | 38.6 | done | 2026-06-22 23:03 +08 |
| Gemma 4 E4B · llama.cpp · Q4_K_M + MTP · conc 1 | llama.cpp + MTP (Google assistant drafter) | Q4_K_M | 65536 | 1 | 99.6 | done | 2026-06-22 21:10 +08 |
| Qwen3.6-35B-A3B · vLLM · FP8 + MTP · conc 1 | vLLM + MTP | FP8 | 65536 | 1 | 54.0 | done | 2026-06-22 20:24 +08 |
| Qwen3.6-27B · vLLM · FP8 + MTP · conc 1 | vLLM + MTP | FP8 | 65536 | 1 | 15.2 | done | 2026-06-22 19:55 +08 |
| Ornith-1.0-35B AEON Uncensored · vLLM (aeon-ultimate) · NVFP4 + DFlash · conc 1 — BLOCKED | vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash, num_speculative_tokens 6) — BLOCKED at KV unification | NVFP4 (compressed-tensors, nvfp4-pack-quantized) | 262144 | 1 | — | blocked | 2026-06-28 16:13 +0800 |
| Qwen3.6-27B (dense) · DDTree (Diffusion Draft Tree) · BLOCKED (hybrid GDN cache) | DDTree research harness (github.com/liranringel/ddtree, PyTorch + transformers, batch-1) + DDTree — tree draft over the z-lab/Qwen3.6-27B-DFlash block-diffusion drafter | BF16 (harness loads the unquantized target via AutoModelForCausalLM) | 4096 | 1 | — | blocked | 2026-07-01 |
| Qwen3.6-35B-A3B · DDTree (Diffusion Draft Tree) · BLOCKED (hybrid GDN cache) | DDTree research harness (github.com/liranringel/ddtree, PyTorch + transformers, batch-1) + DDTree — tree draft over the z-lab/Qwen3.6-35B-A3B-DFlash block-diffusion drafter | BF16 (harness loads the unquantized target via AutoModelForCausalLM) | 4096 | 1 | — | blocked | 2026-07-01 |
conc-2 (15)
| Configuration | Engine | Quant | Ctx | Conc | Decode tok/s | Status | Completed |
|---|---|---|---|---|---|---|---|
| Qwen3.8-27B · vLLM · NVFP4 + MTP · conc-2 | vLLM + MTP | NVFP4 | 65536 | 2 | 39.0 | done | 2026-08-21 01:06 +0800 |
| Qwen3.8-27B · vLLM · NVFP4 · conc-2 | vLLM | NVFP4 | 65536 | 2 | 20.1 | done | 2026-08-21 01:06 +0800 |
| Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-2 | SGLang + DFlash2 | NVFP4 | 262144 | 2 | 56.2 | done | 2026-08-20 05:16 +0800 |
| Qwen3.8-27B · SGLang · NVFP4 + DSpark · conc-2 | SGLang + DSpark | NVFP4 | 262144 | 2 | 43.4 | done | 2026-08-20 03:46 +0800 |
| DiffusionGemma-26B-A4B · vLLM · NVFP4 · conc 2 | vLLM | NVFP4 | 65536 | 2 | 155.7 | done | 2026-07-04 00:33 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 · conc 2 | vLLM + EAGLE3 | NVFP4 | 65536 | 2 | 88.2 | done | 2026-07-03 17:54 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + MTP · conc 2 | vLLM + MTP (Google assistant drafter) | NVFP4 | 65536 | 2 | 97.9 | done | 2026-07-03 16:32 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 · conc 2 | vLLM | NVFP4 | 65536 | 2 | 62.0 | done | 2026-07-03 15:15 +0800 |
| Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 2 | vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) | NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates) | 65536 | 2 | 151.3 | done | 2026-07-02 07:21 +0800 |
| Qwen3.6-27B · vLLM · NVFP4 + MTP · conc 2 | vLLM + MTP (num_speculative_tokens=3) | NVFP4 | 65536 | 2 | 35.7 | done | 2026-07-01 23:14 +0800 |
| Qwen3.6-27B · vLLM · NVFP4 · conc 2 | vLLM | NVFP4 | 65536 | 2 | 18.1 | done | 2026-07-01 22:00 +0800 |
| gpt-oss-20b · vLLM · MXFP4 + EAGLE3 · conc 2 | vLLM + EAGLE3 | MXFP4 | 65536 | 2 | 59.0 | done | 2026-07-01 16:02 +0800 |
| gpt-oss-20b · vLLM · MXFP4 · conc 2 | vLLM | MXFP4 | 65536 | 2 | 83.4 | done | 2026-07-01 15:22 +0800 |
| Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 2 | vLLM | NVFP4 | 65536 | 2 | 113.2 | done | 2026-07-01 14:07 +0800 |
| Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP · conc 2 | vLLM + MTP | NVFP4 | 65536 | 2 | 161.2 | done | 2026-07-01 12:20 +0800 |
conc-4 (15)
| Configuration | Engine | Quant | Ctx | Conc | Decode tok/s | Status | Completed |
|---|---|---|---|---|---|---|---|
| Qwen3.8-27B · vLLM · NVFP4 + MTP · conc-4 | vLLM + MTP | NVFP4 | 65536 | 4 | 70.7 | done | 2026-08-21 01:49 +0800 |
| Qwen3.8-27B · vLLM · NVFP4 · conc-4 | vLLM | NVFP4 | 65536 | 4 | 38.6 | done | 2026-08-21 01:49 +0800 |
| Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-4 | SGLang + DFlash2 | NVFP4 | 262144 | 4 | 85.1 | done | 2026-08-20 05:24 +0800 |
| Qwen3.8-27B · SGLang · NVFP4 + DSpark · conc-4 | SGLang + DSpark | NVFP4 | 262144 | 4 | 75.5 | done | 2026-08-20 04:17 +0800 |
| DiffusionGemma-26B-A4B · vLLM · NVFP4 · conc 4 | vLLM | NVFP4 | 65536 | 4 | 184.1 | done | 2026-07-04 00:33 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 · conc 4 | vLLM + EAGLE3 | NVFP4 | 65536 | 4 | 143.9 | done | 2026-07-03 17:54 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + MTP · conc 4 | vLLM + MTP (Google assistant drafter) | NVFP4 | 65536 | 4 | 165.4 | done | 2026-07-03 16:32 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 · conc 4 | vLLM | NVFP4 | 65536 | 4 | 104.9 | done | 2026-07-03 15:15 +0800 |
| Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 4 | vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) | NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates) | 65536 | 4 | 210.7 | done | 2026-07-02 07:21 +0800 |
| Qwen3.6-27B · vLLM · NVFP4 + MTP · conc 4 | vLLM + MTP (num_speculative_tokens=3) | NVFP4 | 65536 | 4 | 67.8 | done | 2026-07-01 23:35 +0800 |
| Qwen3.6-27B · vLLM · NVFP4 · conc 4 | vLLM | NVFP4 | 65536 | 4 | 35.5 | done | 2026-07-01 22:16 +0800 |
| gpt-oss-20b · vLLM · MXFP4 + EAGLE3 · conc 4 | vLLM + EAGLE3 | MXFP4 | 65536 | 4 | 85.4 | done | 2026-07-01 16:15 +0800 |
| gpt-oss-20b · vLLM · MXFP4 · conc 4 | vLLM | MXFP4 | 65536 | 4 | 127.3 | done | 2026-07-01 15:22 +0800 |
| Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 4 | vLLM | NVFP4 | 65536 | 4 | 173.8 | done | 2026-07-01 14:24 +0800 |
| Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP · conc 4 | vLLM + MTP | NVFP4 | 65536 | 4 | 232.4 | done | 2026-07-01 12:37 +0800 |
conc-8 (30)
| Configuration | Engine | Quant | Ctx | Conc | Decode tok/s | Status | Completed |
|---|---|---|---|---|---|---|---|
| Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-8 | SGLang + DFlash2 | NVFP4 | 262144 | 8 | — | blocked | 2026-08-20 05:25 +0800 |
| Qwen3.8-27B · SGLang · NVFP4 + DSpark · conc-8 | SGLang + DSpark | NVFP4 | 262144 | 8 | 118.3 | done | 2026-08-20 04:46 +0800 |
| Qwen3.8-27B · vLLM · NVFP4 + MTP · conc-8 | vLLM + MTP | NVFP4 | 65536 | 8 | 126.7 | done | 2026-08-15 12:39 +0800 |
| Qwen3.8-27B · vLLM · NVFP4 · conc-8 | vLLM | NVFP4 | 65536 | 8 | 75.0 | done | 2026-08-15 12:39 +0800 |
| Nemotron-3 Puzzle 75B-A9B · vLLM · NVFP4 · conc 8 | vLLM | NVFP4 | 65536 | 8 | 94.0 | done | 2026-07-11 16:54 +08 |
| DiffusionGemma-26B-A4B · vLLM · NVFP4 · conc 8 | vLLM | NVFP4 | 65536 | 8 | 199.2 | done | 2026-07-04 00:33 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 · conc 8 | vLLM + EAGLE3 | NVFP4 | 65536 | 8 | 230.2 | done | 2026-07-03 17:54 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + MTP · conc 8 | vLLM + MTP (Google assistant drafter) | NVFP4 | 65536 | 8 | 267.1 | done | 2026-07-03 16:32 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 · conc 8 | vLLM | NVFP4 | 65536 | 8 | 171.7 | done | 2026-07-03 15:15 +0800 |
| Gemma 4 31B · vLLM · NVFP4 + MTP · conc 8 | vLLM + MTP (Google assistant drafter) | NVFP4 | 65536 | 8 | 117.1 | done | 2026-07-02 15:38 +0800 |
| gpt-oss-20b · vLLM · MXFP4 · conc 8 | vLLM | MXFP4 | 65536 | 8 | 212.5 | done | 2026-07-02 07:51 +0800 |
| Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 8 | vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) | NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates) | 65536 | 8 | 267.4 | done | 2026-07-02 07:21 +0800 |
| Qwen3.6-27B · vLLM · NVFP4 · conc 8 | vLLM | NVFP4 | 65536 | 8 | 67.1 | done | 2026-07-01 22:32 +0800 |
| gpt-oss-120b · vLLM · MXFP4 + EAGLE3 (LMSYS draft) · conc 8 | vLLM + EAGLE3 (lmsys/EAGLE3-gpt-oss-120b-bf16 — the SGLang/SpecForge draft, on vLLM) | MXFP4 | 65536 | 8 | 175.3 | done | 2026-07-01 19:40 +0800 |
| Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 8 | vLLM | NVFP4 | 65536 | 8 | 241.8 | done | 2026-07-01 14:39 +0800 |
| Qwen3.5-122B-A10B · vLLM · int4-AutoRound-EC + DFlash · conc 8 | vLLM + DFlash (z-lab drafter, rev 6c7242c pinned) | int4-AutoRound-EC | 262144 | 8 | 107.4 | done | 2026-06-24 08:07 +0800 |
| Qwen3.6-27B AEON Uncensored · vLLM-ultimate (custom) · NVFP4 + DFlash · conc 8 | vLLM (aeon-vllm-ultimate custom container) + DFlash (z-lab drafter, num_speculative_tokens 12) | NVFP4 (XS mixed-precision) | 65536 | 8 | 127.1 | done | 2026-06-24 01:11 +0800 |
| Qwen3.5-122B-A10B · vLLM · int4-AutoRound-EC · conc 8 | vLLM | int4-AutoRound-EC | 262144 | 8 | 85.5 | done | 2026-06-23 14:54 +08 |
| Qwen3.6-27B AEON Uncensored · vLLM · NVFP4 + MTP (XS) · conc 8 | vLLM + MTP (qwen3_5_mtp, grafted) | NVFP4 (XS mixed-precision) | 65536 | 8 | 127.5 | done | 2026-06-23 14:11 +08 |
| Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP · conc 8 | vLLM + MTP | NVFP4 | 65536 | 8 | 289.1 | done | 2026-06-23 12:49 +08 |
| Qwen3.6-27B · vLLM · NVFP4 + MTP · conc 8 | vLLM + MTP | NVFP4 | 65536 | 8 | 109.1 | done | 2026-06-23 10:33 +08 |
| Gemma 4 31B · vLLM · NVFP4 + EAGLE3 · conc 8 | vLLM + EAGLE3 | NVFP4 | 65536 | 8 | 91.5 | done | 2026-06-23 01:13 +08 |
| Gemma 4 31B · llama.cpp · Q4_K_M + MTP · conc 8 | llama.cpp + MTP (Google assistant drafter) | Q4_K_M | 65536 | 8 | 62.2 | done | 2026-06-23 00:24 +08 |
| Gemma 4 12B · llama.cpp · Q4_K_M + MTP · conc 8 | llama.cpp + MTP (Google assistant drafter) | Q4_K_M | 65536 | 8 | 145.5 | done | 2026-06-23 00:12 +08 |
| Gemma 4 E4B · llama.cpp · Q4_K_M + MTP · conc 8 | llama.cpp + MTP (Google assistant drafter) | Q4_K_M | 65536 | 8 | 222.5 | done | 2026-06-22 23:59 +08 |
| gpt-oss-120b · vLLM · MXFP4 + EAGLE3 · conc 8 | vLLM + EAGLE3 | MXFP4 | 65536 | 8 | 50.0 | done | 2026-06-22 23:44 +08 |
| gpt-oss-20b · vLLM · MXFP4 + EAGLE3 · conc 8 | vLLM + EAGLE3 | MXFP4 | 65536 | 8 | 126.3 | done | 2026-06-22 23:13 +08 |
| Qwen3.6-35B-A3B · vLLM · FP8 + MTP · conc 8 | vLLM + MTP | FP8 | 65536 | 8 | 190.3 | done | 2026-06-22 20:11 +08 |
| Qwen3.6-27B · vLLM · FP8 + MTP · conc 8 | vLLM + MTP | FP8 | 65536 | 8 | 98.0 | done | 2026-06-22 19:42 +08 |
| DeepSeek V4-Flash · llama.cpp · IQ2_XXS-XL (GGUF) | llama.cpp | IQ2_XXS-XL | 8192 | 8 | — | blocked | 2026-06-23 |
| Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-8 | SGLang + DFlash2 | NVFP4 | 262144 | 8 | — | blocked | 2026-08-20 05:25 +0800 |
conc-16 (15)
| Configuration | Engine | Quant | Ctx | Conc | Decode tok/s | Status | Completed |
|---|---|---|---|---|---|---|---|
| Qwen3.8-27B · vLLM · NVFP4 + MTP · conc-16 | vLLM + MTP | NVFP4 | 65536 | 16 | 195.0 | done | 2026-08-21 02:33 +0800 |
| Qwen3.8-27B · vLLM · NVFP4 · conc-16 | vLLM | NVFP4 | 65536 | 16 | 128.6 | done | 2026-08-21 02:33 +0800 |
| Qwen3.8-27B · SGLang · NVFP4 + DSpark · conc-16 | SGLang + DSpark | NVFP4 | 262144 | 16 | — | blocked | 2026-08-20 05:25 +0800 |
| Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-16 | SGLang + DFlash2 | NVFP4 | 262144 | 16 | — | blocked | 2026-08-20 05:25 +0800 |
| DiffusionGemma-26B-A4B · vLLM · NVFP4 · conc 16 | vLLM | NVFP4 | 65536 | 16 | 199.3 | done | 2026-07-04 00:33 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 · conc 16 | vLLM + EAGLE3 | NVFP4 | 65536 | 16 | 376.3 | done | 2026-07-03 17:54 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + MTP · conc 16 | vLLM + MTP (Google assistant drafter) | NVFP4 | 65536 | 16 | 436.7 | done | 2026-07-03 16:32 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 · conc 16 | vLLM | NVFP4 | 65536 | 16 | 273.6 | done | 2026-07-03 15:15 +0800 |
| Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 16 | vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) | NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates) | 65536 | 16 | 340.2 | done | 2026-07-02 07:21 +0800 |
| Qwen3.6-27B · vLLM · NVFP4 + MTP · conc 16 | vLLM + MTP (num_speculative_tokens=3) | NVFP4 | 65536 | 16 | 192.2 | done | 2026-07-01 23:54 +0800 |
| Qwen3.6-27B · vLLM · NVFP4 · conc 16 | vLLM | NVFP4 | 65536 | 16 | 116.6 | done | 2026-07-01 22:58 +0800 |
| gpt-oss-20b · vLLM · MXFP4 + EAGLE3 · conc 16 | vLLM + EAGLE3 | MXFP4 | 65536 | 16 | 431.8 | done | 2026-07-01 16:36 +0800 |
| gpt-oss-20b · vLLM · MXFP4 · conc 16 | vLLM | MXFP4 | 65536 | 16 | 340.7 | done | 2026-07-01 15:35 +0800 |
| Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 16 | vLLM | NVFP4 | 65536 | 16 | 332.4 | done | 2026-07-01 14:58 +0800 |
| Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP · conc 16 | vLLM + MTP | NVFP4 | 65536 | 16 | 433.3 | done | 2026-07-01 12:54 +0800 |
| Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-16 | SGLang + DFlash2 | NVFP4 | 262144 | 16 | — | blocked | 2026-08-20 05:25 +0800 |
| Qwen3.8-27B · SGLang · NVFP4 + DSpark · conc-16 | SGLang + DSpark | NVFP4 | 262144 | 16 | — | blocked | 2026-08-20 05:25 +0800 |
conc-32 (100)
| Configuration | Engine | Quant | Ctx | Conc | Decode tok/s | Status | Completed |
|---|---|---|---|---|---|---|---|
| Qwen3.8-27B · SGLang · NVFP4 + DSpark · conc-32 | SGLang + DSpark | NVFP4 | 262144 | 32 | — | blocked | 2026-08-20 05:25 +0800 |
| Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-32 | SGLang + DFlash2 | NVFP4 | 262144 | 32 | — | blocked | 2026-08-20 05:25 +0800 |
| Qwen3.8-27B · vLLM · FP8 + MTP | vLLM + MTP | FP8 | 65536 | 32 | 203.0 | done | 2026-08-15 11:13 +0800 |
| Qwen3.8-27B · vLLM · FP8 (official) | vLLM | FP8 | 65536 | 32 | 143.9 | done | 2026-08-15 10:50 +0800 |
| Qwen3.8-27B · vLLM · NVFP4 + MTP | vLLM + MTP | NVFP4 | 65536 | 32 | 274.7 | done | 2026-08-15 10:23 +0800 |
| Qwen3.8-27B · vLLM · NVFP4 | vLLM | NVFP4 | 65536 | 32 | 210.0 | done | 2026-08-15 10:01 +0800 |
| Nemotron-3 Puzzle 75B-A9B · vLLM · NVFP4 · conc 32 | vLLM | NVFP4 | 65536 | 32 | 177.8 | done | 2026-07-11 17:21 +08 |
| DiffusionGemma-26B-A4B · vLLM · NVFP4 | vLLM | NVFP4 | 65536 | 32 | 200.7 | done | 2026-07-04 00:33 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 | vLLM + EAGLE3 | NVFP4 | 65536 | 32 | 596.3 | done | 2026-07-03 17:54 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + MTP | vLLM + MTP (Google assistant drafter) | NVFP4 | 65536 | 32 | 697.0 | done | 2026-07-03 16:32 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 | vLLM | NVFP4 | 65536 | 32 | 421.1 | done | 2026-07-03 15:15 +0800 |
| Gemma 4 31B · vLLM · NVFP4 + MTP | vLLM + MTP (Google assistant drafter) | NVFP4 | 65536 | 32 | 323.5 | done | 2026-07-02 21:39 +0800 |
| Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 32 | vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) | NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates) | 65536 | 32 | 407.1 | done | 2026-07-02 07:21 +0800 |
| gpt-oss-120b · vLLM · MXFP4 + EAGLE3 (LMSYS draft) · conc 32 | vLLM + EAGLE3 (lmsys/EAGLE3-gpt-oss-120b-bf16 — the SGLang/SpecForge draft, on vLLM) | MXFP4 | 65536 | 32 | 246.7 | done | 2026-07-01 19:21 +0800 |
| Ornith-1.0-35B AEON Uncensored · vLLM (aeon-ultimate) · NVFP4 · conc 32 | vLLM (aeon-vllm-ultimate custom container) + none (DFlash OFF — see the dflash-blocked page; DFlash trips hybrid+draft KV unification) | NVFP4 (compressed-tensors, nvfp4-pack-quantized) | 32768 | 32 | 422.1 | done | 2026-06-28 16:13 +0800 |
| DeepSeek V4-Flash REAP-K180 · ds4 · NVFP4-hybrid · conc 32 (32K ctx) | ds4 (ds4-nvfp4-spark, custom GB10 runtime) + none — ds4 is a serial single-worker; the DeepSeek MTP head is not exercised by this runtime | NVFP4-hybrid (NVFP4 experts + Q2_K + Q8_0 + F16) | 32768 | 32 | 11.0 | done | 2026-06-28 13:34 +08 |
| MiniMax-M2.7-REAP 172B · vLLM · NVFP4 (W4A4) | vLLM | NVFP4 (W4A4) | 65536 | 32 | 111.9 | done | 2026-06-27 19:22 +0800 |
| MiniMax-M2.5-REAP 139B · vLLM · NVFP4 (W4A16) | vLLM | NVFP4 (W4A16) | 65536 | 32 | 120.0 | done | 2026-06-26 22:25 +0800 |
| GLM-4.5-Air-REAP 82B · vLLM · NVFP4 (W4A16) | vLLM | NVFP4 (W4A16) | 65536 | 32 | 158.4 | done | 2026-06-26 16:10 +0800 |
| MiniMax-M2.7 · llama.cpp · UD-Q3_K_XL | llama.cpp | UD-Q3_K_XL | 65536 | 32 | 58.8 | done | 2026-06-24 06:30 +0800 |
| Qwen3.6-27B AEON Uncensored · vLLM-ultimate (custom) · NVFP4 + DFlash · conc 32 | vLLM (aeon-vllm-ultimate custom container) + DFlash (z-lab drafter, num_speculative_tokens 12) | NVFP4 (XS mixed-precision) | 65536 | 32 | 184.2 | done | 2026-06-24 01:35 +0800 |
| Gemma 4 12B · SGLang · NVFP4 + MTP · AxionML | SGLang + MTP (NEXTN + Google assistant drafter) | NVFP4 | 65536 | 32 | 399.8 | done | 2026-06-24 00:08 +0800 |
| Gemma 4 12B · SGLang · NVFP4 · AxionML | SGLang | NVFP4 | 65536 | 32 | 386.6 | done | 2026-06-23 23:54 +0800 |
| Gemma 4 12B · vLLM · NVFP4 + MTP · RedHatAI | vLLM + MTP (Google assistant drafter) | NVFP4 | 65536 | 32 | 782.4 | done | 2026-06-23 23:35 +0800 |
| Gemma 4 12B · vLLM · NVFP4 · RedHatAI | vLLM | NVFP4 | 65536 | 32 | 503.8 | done | 2026-06-23 23:07 +0800 |
| MiniMax-M2.7 · llama.cpp · UD-IQ4_XS | llama.cpp | UD-IQ4_XS | 32768 | 32 | 58.4 | done | 2026-06-23 22:37 +0800 |
| gpt-oss-120b · SGLang · MXFP4 + EAGLE3 · conc 32 | SGLang + EAGLE3 | MXFP4 | 65536 | 32 | 171.9 | done | 2026-06-23 21:20 +0800 |
| Qwen3.6-27B AEON Uncensored · vLLM · NVFP4 + MTP (XS) | vLLM + MTP (qwen3_5_mtp, grafted) | NVFP4 (XS mixed-precision) | 65536 | 32 | 303.3 | done | 2026-06-23 13:56 +08 |
| Qwen3.6-35B-A3B · SGLang · NVFP4 | SGLang | NVFP4 | 65536 | 32 | — | blocked | 2026-06-23 13:08 +08 |
| Qwen3.6-35B-A3B · SGLang · NVFP4 + MTP | SGLang + MTP (NEXTN) | NVFP4 | 65536 | 32 | — | blocked | 2026-06-23 13:08 +08 |
| Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP | vLLM + MTP | NVFP4 | 65536 | 32 | 541.3 | done | 2026-06-23 12:34 +08 |
| Qwen3.6-27B · SGLang · NVFP4 + MTP | SGLang + MTP (NEXTN) | NVFP4 | 65536 | 32 | 196.6 | done | 2026-06-23 12:18 +08 |
| Qwen3.6-27B · SGLang · NVFP4 | SGLang | NVFP4 | 65536 | 32 | 178.0 | done | 2026-06-23 11:35 +08 |
| Qwen3.6-35B-A3B · vLLM · NVFP4 | vLLM | NVFP4 | 65536 | 32 | 430.8 | done | 2026-06-23 11:13 +08 |
| Qwen3.6-27B · vLLM · NVFP4 + MTP | vLLM + MTP | NVFP4 | 65536 | 32 | 274.1 | done | 2026-06-23 10:19 +08 |
| Qwen3.6-27B · vLLM · NVFP4 | vLLM | NVFP4 | 65536 | 32 | 187.7 | done | 2026-06-23 09:54 +08 |
| Gemma 4 E4B · vLLM · FP8 + MTP | vLLM + MTP (Google assistant drafter) | FP8 | 65536 | 32 | 1261.5 | done | 2026-06-23 09:19 +08 |
| Gemma 4 E4B · vLLM · NVFP4 | vLLM | NVFP4 | 65536 | 32 | 1073.8 | done | 2026-06-22 21:57 +08 |
| Qwen3.6-35B-A3B · vLLM · FP8 + MTP | vLLM + MTP | FP8 | 65536 | 32 | 407.9 | done | 2026-06-22 19:28 +08 |
| Qwen3.6-35B-A3B · vLLM · FP8 | vLLM | FP8 | 65536 | 32 | 286.0 | done | 2026-06-22 19:09 +08 |
| Qwen3.6-27B · vLLM · FP8 + MTP | vLLM + MTP | FP8 | 65536 | 32 | 240.9 | done | 2026-06-22 18:46 +08 |
| Qwen3.6-27B · vLLM · FP8 | vLLM | FP8 | 65536 | 32 | 154.7 | done | 2026-06-22 18:11 +08 |
| gpt-oss-20b · vLLM · MXFP4 + EAGLE3 | vLLM + EAGLE3 | MXFP4 | 65536 | 32 | 686.5 | done | 2026-06-22 17:52 +08 |
| gpt-oss-120b · vLLM · MXFP4 + EAGLE3 | vLLM + EAGLE3 | MXFP4 | 65536 | 32 | 138.5 | done | 2026-06-22 17:42 +08 |
| Llama 4 Scout 17B-16E · vLLM · AWQ-Int4 | vLLM | W4A16 | 65536 | 32 | 61.5 | done | 2026-06-22 17:15 +08 |
| Qwen3-Coder-Next · vLLM · FP8 | vLLM | FP8 | 262144 | 32 | 184.7 | done | 2026-06-22 16:54 +08 |
| DeepSeek-R1-Distill-Llama-70B · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 52.8 | done | 2026-06-22 16:30 +08 |
| Llama 3.3 70B · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 48.7 | done | 2026-06-22 16:01 +08 |
| gpt-oss-120b · vLLM · MXFP4 | vLLM | MXFP4 | 65536 | 32 | 252.8 | done | 2026-06-22 15:30 +08 |
| Gemma 4 E4B · llama.cpp · Q4_K_M + MTP | llama.cpp + MTP (Google assistant drafter) | Q4_K_M | 65536 | 32 | 276.6 | done | 2026-06-22 12:58 +08 |
| Gemma 4 E4B · vLLM · BF16 | vLLM | BF16 | 65536 | 32 | 565.8 | done | 2026-06-22 12:23 +08 |
| Gemma 4 12B · llama.cpp · Q6_K | llama.cpp | Q6_K | 65536 | 32 | 159.3 | done | 2026-06-22 12:09 +08 |
| Gemma 4 12B · llama.cpp · Q4_K_M + MTP | llama.cpp + MTP (Google assistant drafter) | Q4_K_M | 65536 | 32 | 202.2 | done | 2026-06-22 11:52 +08 |
| Gemma 4 31B · llama.cpp · Q4_K_M + MTP | llama.cpp + MTP (Google assistant drafter) | Q4_K_M | 65536 | 32 | 93.0 | done | 2026-06-22 11:36 +08 |
| Gemma 4 12B · llama.cpp · Q8_0 | llama.cpp | Q8_0 | 65536 | 32 | 153.0 | done | 2026-06-22 11:18 +08 |
| Gemma 4 E4B · vLLM · FP8 | vLLM | FP8 | 65536 | 32 | 869.7 | done | 2026-06-22 11:00 +08 |
| Gemma 4 31B · vLLM · NVFP4 + EAGLE3 | vLLM + EAGLE3 | NVFP4 | 65536 | 32 | 264.7 | done | 2026-06-22 10:51 +08 |
| Gemma 4 31B · vLLM · FP8 | vLLM | FP8 | 65536 | 32 | 147.6 | done | 2026-06-22 10:10 +08 |
| Gemma 4 26B-A4B · vLLM · FP8 | vLLM | FP8 | 65536 | 32 | 316.9 | done | 2026-06-22 09:47 +08 |
| Gemma 4 31B · vLLM · NVFP4 | vLLM | NVFP4 | 65536 | 32 | 167.0 | done | 2026-06-22 09:12 +08 |
| DeepSeek-R1-Distill-Qwen-32B · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 117.6 | done | 2026-06-22 08:49 +08 |
| Gemma 4 31B · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 78.5 | done | 2026-06-22 08:29 +08 |
| Gemma 4 26B-A4B · vLLM · BF16 | vLLM | BF16 | 65536 | 32 | 190.1 | done | 2026-06-22 08:10 +08 |
| Granite 4.1 30B · vLLM · FP8 | vLLM | FP8 | 65536 | 32 | 181.8 | done | 2026-06-22 07:00 +08 |
| Qwen3-Coder-30B-A3B · vLLM · FP8 | vLLM | FP8 | 65536 | 32 | 295.8 | done | 2026-06-22 06:39 +08 |
| gpt-oss-20b · vLLM · MXFP4 | vLLM | MXFP4 | 65536 | 32 | 535.3 | done | 2026-06-22 06:00 +08 |
| DeepSeek-Coder-V2-Lite-Instruct · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 130.5 | done | 2026-06-22 05:43 +08 |
| Gemma 4 E4B · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 435.0 | done | 2026-06-22 05:27 +08 |
| Phi-4-reasoning-plus · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 231.0 | done | 2026-06-22 05:16 +08 |
| Ministral-3-14B-Reasoning · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 266.0 | done | 2026-06-22 05:00 +08 |
| DeepSeek-R1-Distill-Qwen-14B · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 243.8 | done | 2026-06-22 04:43 +08 |
| Gemma 4 12B · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 195.3 | done | 2026-06-22 04:27 +08 |
| DeepSeek-R1-0528-Qwen3-8B · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 374.7 | done | 2026-06-22 04:10 +08 |
| Llama 3.1 8B · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 365.2 | done | 2026-06-22 03:59 +08 |
| Granite 4.1 8B · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 323.8 | done | 2026-06-22 03:48 +08 |
| Ministral-3-3B · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 523.7 | done | 2026-06-22 03:35 +08 |
| Phi-4-mini-reasoning · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 552.4 | done | 2026-06-22 03:24 +08 |
| Granite 4.1 3B · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 617.5 | done | 2026-06-22 03:16 +08 |
| Nemotron-3 Nano-4B · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 141.3 | done | 2026-06-22 03:10 +08 |
| Mistral Small 4 119B · vLLM · NVFP4 | vLLM | NVFP4 | 65536 | 32 | 135.0 | done | 2026-06-22 02:50 +08 |
| Nemotron-Terminal-32B · vLLM · FP8 | vLLM | FP8 | 40960 | 32 | 166.2 | done | 2026-06-22 01:28 +08 |
| Nemotron-3 Super 120B · vLLM · NVFP4 | vLLM | NVFP4 | 65536 | 32 | 96.9 | done | 2026-06-22 00:22 +08 |
| Nemotron-3 Nano-Omni 30B-A3B · vLLM · NVFP4 | vLLM | NVFP4 | 65536 | 32 | 388.9 | done | 2026-06-21 23:43 +08 |
| Nemotron-3 Elastic 30B-A3B · vLLM · NVFP4 | vLLM | NVFP4 | 65536 | 32 | 353.0 | done | 2026-06-21 23:23 +08 |
| gpt-oss-120b · SGLang · MXFP4 | SGLang | MXFP4 | 65536 | 32 | 140.3 | done | 2026-06-21 22:56 +08 |
| gpt-oss-20b · SGLang · MXFP4 | SGLang | MXFP4 | 65536 | 32 | 279.5 | done | 2026-06-21 22:08 +08 |
| SmolLM3-3B · llama.cpp · Q8_0 | llama.cpp | Q8_0 | 65536 | 32 | 570.6 | done | 2026-06-21 21:20 +08 |
| SmolLM3-3B · llama.cpp · Q4_K_M | llama.cpp | Q4_K_M | 65536 | 32 | 653.6 | done | 2026-06-21 21:12 +08 |
| DeepSeek V4-Flash · vLLM · AWQ-Int4 | vLLM | NVFP4 | 131072 | 32 | — | blocked | — |
| DeepSeek V4-Flash · vLLM · NVFP4 + EAGLE3.1 | vLLM + EAGLE3.1 | NVFP4 | 65536 | 32 | — | blocked | — |
| GLM-4.5-Air · vLLM · (quant TBD) | vLLM | TBD | 65536 | 32 | — | blocked | — |
| GLM-4.7-Flash · vLLM · FP8 | vLLM | FP8 | 131072 | 32 | — | blocked | — |
| Gemma 4 12B · vLLM · NVFP4 + EAGLE3 | vLLM + EAGLE3 | NVFP4 | 65536 | 32 | — | blocked | 2026-07-02 |
| Gemma 4 E4B · vLLM · FP8 + EAGLE3 | vLLM + EAGLE3 (RedHatAI speculator convention) | FP8 | 65536 | 32 | — | blocked | 2026-07-02 |
| Gemma 4 E4B · vLLM · NVFP4 + MTP | vLLM + MTP (Google assistant drafter) | NVFP4 | 65536 | 32 | — | blocked | 2026-06-23 |
| Granite-switch 4.1 30B · vLLM · FP8 | vLLM | FP8 | 131072 | 32 | — | blocked | — |
| Llama 4 Maverick 402B · llama.cpp · UD-Q2_K_XL | llama.cpp | UD-Q2_K_XL | 131072 | 32 | — | blocked | — |
| MiniMax-M2.7 · llama.cpp · MXFP4_MOE | llama.cpp | MXFP4_MOE | 65536 | 32 | — | blocked | — |
| MiniMax-M2.7 · vLLM · NVFP4 | vLLM | NVFP4 | 65536 | 32 | — | blocked | — |
| Qwen3.6-35B-A3B · SGLang · NVFP4 | SGLang | NVFP4 | 65536 | 32 | — | blocked | 2026-06-23 13:08 +08 |
| Qwen3.6-35B-A3B · SGLang · NVFP4 + MTP | SGLang + MTP (NEXTN) | NVFP4 | 65536 | 32 | — | blocked | 2026-06-23 13:08 +08 |
| Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-32 | SGLang + DFlash2 | NVFP4 | 262144 | 32 | — | blocked | 2026-08-20 05:25 +0800 |
| Qwen3.8-27B · SGLang · NVFP4 + DSpark · conc-32 | SGLang + DSpark | NVFP4 | 262144 | 32 | — | blocked | 2026-08-20 05:25 +0800 |
| gpt-oss-20b · llama.cpp · MXFP4 | llama.cpp | MXFP4 | 131072 | 32 | — | blocked | 2026-06-21 |
conc-64 (8)
| Configuration | Engine | Quant | Ctx | Conc | Decode tok/s | Status | Completed |
|---|---|---|---|---|---|---|---|
| Qwen3.8-27B · vLLM · NVFP4 + MTP · conc-64 | vLLM + MTP | NVFP4 | 65536 | 64 | 324.1 | done | 2026-08-21 03:13 +0800 |
| Qwen3.8-27B · vLLM · NVFP4 · conc-64 | vLLM | NVFP4 | 65536 | 64 | 283.7 | done | 2026-08-21 03:13 +0800 |
| Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 64 | vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) | NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates) | 65536 | 64 | 420.4 | done | 2026-07-04 14:11 +0800 |
| Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 64 | vLLM | NVFP4 | 65536 | 64 | 547.9 | done | 2026-07-04 13:41 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 · conc 64 | vLLM + EAGLE3 | NVFP4 | 65536 | 64 | 889.0 | done | 2026-07-04 13:11 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 · conc 64 | vLLM | NVFP4 | 65536 | 64 | 680.9 | done | 2026-07-04 12:42 +0800 |
| Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP · conc 64 | vLLM + MTP | NVFP4 | 65536 | 64 | 670.8 | done | 2026-07-04 12:09 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + MTP · conc 64 | vLLM + MTP (Google assistant drafter) | NVFP4 | 65536 | 64 | 1027.5 | done | 2026-07-04 10:25 +0800 |
conc-128 (8)
| Configuration | Engine | Quant | Ctx | Conc | Decode tok/s | Status | Completed |
|---|---|---|---|---|---|---|---|
| Qwen3.8-27B · vLLM · NVFP4 + MTP · conc-128 | vLLM + MTP | NVFP4 | 65536 | 128 | 351.2 | done | 2026-08-21 03:48 +0800 |
| Qwen3.8-27B · vLLM · NVFP4 · conc-128 | vLLM | NVFP4 | 65536 | 128 | 346.2 | done | 2026-08-21 03:48 +0800 |
| Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 128 (memory ceiling — watchdog trip) | vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) | NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates) | 65536 | 128 | 425.7 | done | 2026-07-04 14:17 +0800 |
| Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 128 | vLLM | NVFP4 | 65536 | 128 | 675.9 | done | 2026-07-04 13:50 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 · conc 128 | vLLM + EAGLE3 | NVFP4 | 65536 | 128 | 1210.2 | done | 2026-07-04 13:20 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 · conc 128 | vLLM | NVFP4 | 65536 | 128 | 995.1 | done | 2026-07-04 12:52 +0800 |
| Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP · conc 128 (the knee) | vLLM + MTP | NVFP4 | 65536 | 128 | 749.9 | done | 2026-07-04 12:31 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + MTP · conc 128 | vLLM + MTP (Google assistant drafter) | NVFP4 | 65536 | 128 | 1380.1 | done | 2026-07-04 10:32 +0800 |
conc-256 (3)
| Configuration | Engine | Quant | Ctx | Conc | Decode tok/s | Status | Completed |
|---|---|---|---|---|---|---|---|
| Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 · conc 256 (memory ceiling — watchdog trip) | vLLM + EAGLE3 | NVFP4 | 65536 | 256 | 2.0 | done | 2026-07-04 13:26 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 · conc 256 | vLLM | NVFP4 | 65536 | 256 | 1365.9 | done | 2026-07-04 13:01 +0800 |
| Gemma 4 26B-A4B · vLLM · NVFP4 + MTP · conc 256 (collapse — the ceiling) | vLLM + MTP (Google assistant drafter) | NVFP4 | 65536 | 256 | 0.2 | done | 2026-07-04 11:04 +0800 |