DGX Spark Autobench

Browse by size bucket

Configurations grouped by size bucket (total parameters). ← all tag kinds

≤4B5-15B16-40B41-130B130B+

≤4B (16)

ConfigurationEngineQuantCtxConcDecode tok/sStatusCompleted
Gemma 4 E4B · vLLM · FP8 + MTP vLLM + MTP (Google assistant drafter) FP8 65536 32 1261.5 done 2026-06-23 09:19 +08
Gemma 4 E4B · llama.cpp · Q4_K_M + MTP · conc 8 llama.cpp + MTP (Google assistant drafter) Q4_K_M 65536 8 222.5 done 2026-06-22 23:59 +08
Gemma 4 E4B · vLLM · NVFP4 vLLM NVFP4 65536 32 1073.8 done 2026-06-22 21:57 +08
Gemma 4 E4B · llama.cpp · Q4_K_M + MTP · conc 1 llama.cpp + MTP (Google assistant drafter) Q4_K_M 65536 1 99.6 done 2026-06-22 21:10 +08
Gemma 4 E4B · llama.cpp · Q4_K_M + MTP llama.cpp + MTP (Google assistant drafter) Q4_K_M 65536 32 276.6 done 2026-06-22 12:58 +08
Gemma 4 E4B · vLLM · BF16 vLLM BF16 65536 32 565.8 done 2026-06-22 12:23 +08
Gemma 4 E4B · vLLM · FP8 vLLM FP8 65536 32 869.7 done 2026-06-22 11:00 +08
Gemma 4 E4B · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 435.0 done 2026-06-22 05:27 +08
Ministral-3-3B · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 523.7 done 2026-06-22 03:35 +08
Phi-4-mini-reasoning · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 552.4 done 2026-06-22 03:24 +08
Granite 4.1 3B · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 617.5 done 2026-06-22 03:16 +08
Nemotron-3 Nano-4B · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 141.3 done 2026-06-22 03:10 +08
SmolLM3-3B · llama.cpp · Q8_0 llama.cpp Q8_0 65536 32 570.6 done 2026-06-21 21:20 +08
SmolLM3-3B · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 653.6 done 2026-06-21 21:12 +08
Gemma 4 E4B · vLLM · FP8 + EAGLE3 vLLM + EAGLE3 (RedHatAI speculator convention) FP8 65536 32 — blocked 2026-07-02
Gemma 4 E4B · vLLM · NVFP4 + MTP vLLM + MTP (Google assistant drafter) NVFP4 65536 32 — blocked 2026-06-23

5-15B (17)

ConfigurationEngineQuantCtxConcDecode tok/sStatusCompleted
Gemma 4 12B · SGLang · NVFP4 + MTP · AxionML SGLang + MTP (NEXTN + Google assistant drafter) NVFP4 65536 32 399.8 done 2026-06-24 00:08 +0800
Gemma 4 12B · SGLang · NVFP4 · AxionML SGLang NVFP4 65536 32 386.6 done 2026-06-23 23:54 +0800
Gemma 4 12B · vLLM · NVFP4 + MTP · RedHatAI vLLM + MTP (Google assistant drafter) NVFP4 65536 32 782.4 done 2026-06-23 23:35 +0800
Gemma 4 12B · vLLM · NVFP4 · RedHatAI vLLM NVFP4 65536 32 503.8 done 2026-06-23 23:07 +0800
Gemma 4 12B · llama.cpp · Q4_K_M + MTP · conc 8 llama.cpp + MTP (Google assistant drafter) Q4_K_M 65536 8 145.5 done 2026-06-23 00:12 +08
Gemma 4 12B · llama.cpp · Q4_K_M + MTP · conc 1 llama.cpp + MTP (Google assistant drafter) Q4_K_M 65536 1 49.2 done 2026-06-23 00:06 +08
Gemma 4 12B · llama.cpp · Q6_K llama.cpp Q6_K 65536 32 159.3 done 2026-06-22 12:09 +08
Gemma 4 12B · llama.cpp · Q4_K_M + MTP llama.cpp + MTP (Google assistant drafter) Q4_K_M 65536 32 202.2 done 2026-06-22 11:52 +08
Gemma 4 12B · llama.cpp · Q8_0 llama.cpp Q8_0 65536 32 153.0 done 2026-06-22 11:18 +08
Phi-4-reasoning-plus · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 231.0 done 2026-06-22 05:16 +08
Ministral-3-14B-Reasoning · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 266.0 done 2026-06-22 05:00 +08
DeepSeek-R1-Distill-Qwen-14B · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 243.8 done 2026-06-22 04:43 +08
Gemma 4 12B · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 195.3 done 2026-06-22 04:27 +08
DeepSeek-R1-0528-Qwen3-8B · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 374.7 done 2026-06-22 04:10 +08
Llama 3.1 8B · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 365.2 done 2026-06-22 03:59 +08
Granite 4.1 8B · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 323.8 done 2026-06-22 03:48 +08
Gemma 4 12B · vLLM · NVFP4 + EAGLE3 vLLM + EAGLE3 NVFP4 65536 32 — blocked 2026-07-02

16-40B (162)

ConfigurationEngineQuantCtxConcDecode tok/sStatusCompleted
Qwen3.8-27B · vLLM · NVFP4 + MTP · conc-128 vLLM + MTP NVFP4 65536 128 351.2 done 2026-08-21 03:48 +0800
Qwen3.8-27B · vLLM · NVFP4 · conc-128 vLLM NVFP4 65536 128 346.2 done 2026-08-21 03:48 +0800
Qwen3.8-27B · vLLM · NVFP4 + MTP · conc-64 vLLM + MTP NVFP4 65536 64 324.1 done 2026-08-21 03:13 +0800
Qwen3.8-27B · vLLM · NVFP4 · conc-64 vLLM NVFP4 65536 64 283.7 done 2026-08-21 03:13 +0800
Qwen3.8-27B · vLLM · NVFP4 + MTP · conc-16 vLLM + MTP NVFP4 65536 16 195.0 done 2026-08-21 02:33 +0800
Qwen3.8-27B · vLLM · NVFP4 · conc-16 vLLM NVFP4 65536 16 128.6 done 2026-08-21 02:33 +0800
Qwen3.8-27B · vLLM · NVFP4 + MTP · conc-4 vLLM + MTP NVFP4 65536 4 70.7 done 2026-08-21 01:49 +0800
Qwen3.8-27B · vLLM · NVFP4 · conc-4 vLLM NVFP4 65536 4 38.6 done 2026-08-21 01:49 +0800
Qwen3.8-27B · vLLM · NVFP4 + MTP · conc-2 vLLM + MTP NVFP4 65536 2 39.0 done 2026-08-21 01:06 +0800
Qwen3.8-27B · vLLM · NVFP4 · conc-2 vLLM NVFP4 65536 2 20.1 done 2026-08-21 01:06 +0800
Qwen3.8-27B · SGLang · NVFP4 + DSpark · conc-32 SGLang + DSpark NVFP4 262144 32 — blocked 2026-08-20 05:25 +0800
Qwen3.8-27B · SGLang · NVFP4 + DSpark · conc-16 SGLang + DSpark NVFP4 262144 16 — blocked 2026-08-20 05:25 +0800
Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-8 SGLang + DFlash2 NVFP4 262144 8 — blocked 2026-08-20 05:25 +0800
Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-32 SGLang + DFlash2 NVFP4 262144 32 — blocked 2026-08-20 05:25 +0800
Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-16 SGLang + DFlash2 NVFP4 262144 16 — blocked 2026-08-20 05:25 +0800
Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-4 SGLang + DFlash2 NVFP4 262144 4 85.1 done 2026-08-20 05:24 +0800
Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-2 SGLang + DFlash2 NVFP4 262144 2 56.2 done 2026-08-20 05:16 +0800
Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-1 SGLang + DFlash2 NVFP4 262144 1 30.0 done 2026-08-20 05:04 +0800
Qwen3.8-27B · SGLang · NVFP4 + DSpark · conc-8 SGLang + DSpark NVFP4 262144 8 118.3 done 2026-08-20 04:46 +0800
Qwen3.8-27B · SGLang · NVFP4 + DSpark · conc-4 SGLang + DSpark NVFP4 262144 4 75.5 done 2026-08-20 04:17 +0800
Qwen3.8-27B · SGLang · NVFP4 + DSpark · conc-2 SGLang + DSpark NVFP4 262144 2 43.4 done 2026-08-20 03:46 +0800
Qwen3.8-27B · SGLang · NVFP4 + DSpark · conc-1 SGLang + DSpark NVFP4 262144 1 23.4 done 2026-08-20 03:31 +0800
Qwen3.8-27B · vLLM · NVFP4 + MTP · conc-8 vLLM + MTP NVFP4 65536 8 126.7 done 2026-08-15 12:39 +0800
Qwen3.8-27B · vLLM · NVFP4 · conc-8 vLLM NVFP4 65536 8 75.0 done 2026-08-15 12:39 +0800
Qwen3.8-27B · vLLM · NVFP4 + MTP · conc-1 vLLM + MTP NVFP4 65536 1 21.4 done 2026-08-15 11:56 +0800
Qwen3.8-27B · vLLM · NVFP4 · conc-1 vLLM NVFP4 65536 1 11.2 done 2026-08-15 11:56 +0800
Qwen3.8-27B · vLLM · FP8 + MTP vLLM + MTP FP8 65536 32 203.0 done 2026-08-15 11:13 +0800
Qwen3.8-27B · vLLM · FP8 (official) vLLM FP8 65536 32 143.9 done 2026-08-15 10:50 +0800
Qwen3.8-27B · vLLM · NVFP4 + MTP vLLM + MTP NVFP4 65536 32 274.7 done 2026-08-15 10:23 +0800
Qwen3.8-27B · vLLM · NVFP4 vLLM NVFP4 65536 32 210.0 done 2026-08-15 10:01 +0800
Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 128 (memory ceiling — watchdog trip) vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates) 65536 128 425.7 done 2026-07-04 14:17 +0800
Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 64 vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates) 65536 64 420.4 done 2026-07-04 14:11 +0800
Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 128 vLLM NVFP4 65536 128 675.9 done 2026-07-04 13:50 +0800
Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 64 vLLM NVFP4 65536 64 547.9 done 2026-07-04 13:41 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 · conc 256 (memory ceiling — watchdog trip) vLLM + EAGLE3 NVFP4 65536 256 2.0 done 2026-07-04 13:26 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 · conc 128 vLLM + EAGLE3 NVFP4 65536 128 1210.2 done 2026-07-04 13:20 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 · conc 64 vLLM + EAGLE3 NVFP4 65536 64 889.0 done 2026-07-04 13:11 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 · conc 256 vLLM NVFP4 65536 256 1365.9 done 2026-07-04 13:01 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 · conc 128 vLLM NVFP4 65536 128 995.1 done 2026-07-04 12:52 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 · conc 64 vLLM NVFP4 65536 64 680.9 done 2026-07-04 12:42 +0800
Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP · conc 128 (the knee) vLLM + MTP NVFP4 65536 128 749.9 done 2026-07-04 12:31 +0800
Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP · conc 64 vLLM + MTP NVFP4 65536 64 670.8 done 2026-07-04 12:09 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + MTP · conc 256 (collapse — the ceiling) vLLM + MTP (Google assistant drafter) NVFP4 65536 256 0.2 done 2026-07-04 11:04 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + MTP · conc 128 vLLM + MTP (Google assistant drafter) NVFP4 65536 128 1380.1 done 2026-07-04 10:32 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + MTP · conc 64 vLLM + MTP (Google assistant drafter) NVFP4 65536 64 1027.5 done 2026-07-04 10:25 +0800
DiffusionGemma-26B-A4B · vLLM · NVFP4 vLLM NVFP4 65536 32 200.7 done 2026-07-04 00:33 +0800
DiffusionGemma-26B-A4B · vLLM · NVFP4 · conc 8 vLLM NVFP4 65536 8 199.2 done 2026-07-04 00:33 +0800
DiffusionGemma-26B-A4B · vLLM · NVFP4 · conc 4 vLLM NVFP4 65536 4 184.1 done 2026-07-04 00:33 +0800
DiffusionGemma-26B-A4B · vLLM · NVFP4 · conc 2 vLLM NVFP4 65536 2 155.7 done 2026-07-04 00:33 +0800
DiffusionGemma-26B-A4B · vLLM · NVFP4 · conc 16 vLLM NVFP4 65536 16 199.3 done 2026-07-04 00:33 +0800
DiffusionGemma-26B-A4B · vLLM · NVFP4 · conc 1 vLLM NVFP4 65536 1 116.0 done 2026-07-04 00:33 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 vLLM + EAGLE3 NVFP4 65536 32 596.3 done 2026-07-03 17:54 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 · conc 8 vLLM + EAGLE3 NVFP4 65536 8 230.2 done 2026-07-03 17:54 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 · conc 4 vLLM + EAGLE3 NVFP4 65536 4 143.9 done 2026-07-03 17:54 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 · conc 2 vLLM + EAGLE3 NVFP4 65536 2 88.2 done 2026-07-03 17:54 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 · conc 16 vLLM + EAGLE3 NVFP4 65536 16 376.3 done 2026-07-03 17:54 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + EAGLE3 · conc 1 vLLM + EAGLE3 NVFP4 65536 1 50.1 done 2026-07-03 17:54 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + MTP vLLM + MTP (Google assistant drafter) NVFP4 65536 32 697.0 done 2026-07-03 16:32 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + MTP · conc 8 vLLM + MTP (Google assistant drafter) NVFP4 65536 8 267.1 done 2026-07-03 16:32 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + MTP · conc 4 vLLM + MTP (Google assistant drafter) NVFP4 65536 4 165.4 done 2026-07-03 16:32 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + MTP · conc 2 vLLM + MTP (Google assistant drafter) NVFP4 65536 2 97.9 done 2026-07-03 16:32 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + MTP · conc 16 vLLM + MTP (Google assistant drafter) NVFP4 65536 16 436.7 done 2026-07-03 16:32 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 + MTP · conc 1 vLLM + MTP (Google assistant drafter) NVFP4 65536 1 57.2 done 2026-07-03 16:32 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 vLLM NVFP4 65536 32 421.1 done 2026-07-03 15:15 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 · conc 8 vLLM NVFP4 65536 8 171.7 done 2026-07-03 15:15 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 · conc 4 vLLM NVFP4 65536 4 104.9 done 2026-07-03 15:15 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 · conc 2 vLLM NVFP4 65536 2 62.0 done 2026-07-03 15:15 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 · conc 16 vLLM NVFP4 65536 16 273.6 done 2026-07-03 15:15 +0800
Gemma 4 26B-A4B · vLLM · NVFP4 · conc 1 vLLM NVFP4 65536 1 29.5 done 2026-07-03 15:15 +0800
Gemma 4 31B · vLLM · NVFP4 + MTP vLLM + MTP (Google assistant drafter) NVFP4 65536 32 323.5 done 2026-07-02 21:39 +0800
Gemma 4 31B · vLLM · NVFP4 + MTP · conc 1 vLLM + MTP (Google assistant drafter) NVFP4 65536 1 18.3 done 2026-07-02 15:52 +0800
Gemma 4 31B · vLLM · NVFP4 + MTP · conc 8 vLLM + MTP (Google assistant drafter) NVFP4 65536 8 117.1 done 2026-07-02 15:38 +0800
gpt-oss-20b · vLLM · MXFP4 · conc 8 vLLM MXFP4 65536 8 212.5 done 2026-07-02 07:51 +0800
gpt-oss-20b · vLLM · MXFP4 · conc 1 vLLM MXFP4 65536 1 45.6 done 2026-07-02 07:51 +0800
Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 1 vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates) 65536 1 99.8 done 2026-07-02 07:21 +0800
Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 8 vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates) 65536 8 267.4 done 2026-07-02 07:21 +0800
Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 4 vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates) 65536 4 210.7 done 2026-07-02 07:21 +0800
Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 32 vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates) 65536 32 407.1 done 2026-07-02 07:21 +0800
Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 2 vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates) 65536 2 151.3 done 2026-07-02 07:21 +0800
Qwen3.6-35B-A3B · vLLM-ultimate (AEON) · NVFP4 + DFlash · conc 16 vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) NVFP4 (modelopt_mixed — W4A16_NVFP4 experts + FP8 GDN gates) 65536 16 340.2 done 2026-07-02 07:21 +0800
Qwen3.6-27B · vLLM · NVFP4 + MTP · conc 16 vLLM + MTP (num_speculative_tokens=3) NVFP4 65536 16 192.2 done 2026-07-01 23:54 +0800
Qwen3.6-27B · vLLM · NVFP4 + MTP · conc 4 vLLM + MTP (num_speculative_tokens=3) NVFP4 65536 4 67.8 done 2026-07-01 23:35 +0800
Qwen3.6-27B · vLLM · NVFP4 + MTP · conc 2 vLLM + MTP (num_speculative_tokens=3) NVFP4 65536 2 35.7 done 2026-07-01 23:14 +0800
Qwen3.6-27B · vLLM · NVFP4 · conc 16 vLLM NVFP4 65536 16 116.6 done 2026-07-01 22:58 +0800
Qwen3.6-27B · vLLM · NVFP4 · conc 8 vLLM NVFP4 65536 8 67.1 done 2026-07-01 22:32 +0800
Qwen3.6-27B · vLLM · NVFP4 · conc 4 vLLM NVFP4 65536 4 35.5 done 2026-07-01 22:16 +0800
Qwen3.6-27B · vLLM · NVFP4 · conc 2 vLLM NVFP4 65536 2 18.1 done 2026-07-01 22:00 +0800
Qwen3.6-27B · vLLM · NVFP4 · conc 1 vLLM NVFP4 65536 1 9.3 done 2026-07-01 21:44 +0800
Qwen3-Coder-30B-A3B · DDTree vs DFlash · HumanEval (coding workload) · single-stream DDTree research harness (github.com/liranringel/ddtree, PyTorch + transformers, batch-1) + DDTree tree draft (budgets 64/256) vs single-line DFlash — z-lab/Qwen3-Coder-30B-A3B-DFlash, block_size 16 BF16 (harness loads the unquantized target via AutoModelForCausalLM) 4096 1 49.3 done 2026-07-01 21:26 +0800
Qwen3-Coder-30B-A3B · DDTree vs DFlash vs autoregressive · single-stream (batch-1) DDTree research harness (github.com/liranringel/ddtree, PyTorch + transformers, batch-1) + DDTree tree draft (budgets 64/256) vs single-line DFlash — z-lab/Qwen3-Coder-30B-A3B-DFlash, block_size 16 BF16 (harness loads the unquantized target via AutoModelForCausalLM) 4096 1 20.8 done 2026-07-01 21:13 +0800
gpt-oss-20b · vLLM · MXFP4 + EAGLE3 · conc 16 vLLM + EAGLE3 MXFP4 65536 16 431.8 done 2026-07-01 16:36 +0800
gpt-oss-20b · vLLM · MXFP4 + EAGLE3 · conc 4 vLLM + EAGLE3 MXFP4 65536 4 85.4 done 2026-07-01 16:15 +0800
gpt-oss-20b · vLLM · MXFP4 + EAGLE3 · conc 2 vLLM + EAGLE3 MXFP4 65536 2 59.0 done 2026-07-01 16:02 +0800
gpt-oss-20b · vLLM · MXFP4 · conc 16 vLLM MXFP4 65536 16 340.7 done 2026-07-01 15:35 +0800
gpt-oss-20b · vLLM · MXFP4 · conc 4 vLLM MXFP4 65536 4 127.3 done 2026-07-01 15:22 +0800
gpt-oss-20b · vLLM · MXFP4 · conc 2 vLLM MXFP4 65536 2 83.4 done 2026-07-01 15:22 +0800
Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 16 vLLM NVFP4 65536 16 332.4 done 2026-07-01 14:58 +0800
Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 8 vLLM NVFP4 65536 8 241.8 done 2026-07-01 14:39 +0800
Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 4 vLLM NVFP4 65536 4 173.8 done 2026-07-01 14:24 +0800
Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 2 vLLM NVFP4 65536 2 113.2 done 2026-07-01 14:07 +0800
Qwen3.6-35B-A3B · vLLM · NVFP4 · conc 1 vLLM NVFP4 65536 1 74.7 done 2026-07-01 13:50 +0800
Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP · conc 16 vLLM + MTP NVFP4 65536 16 433.3 done 2026-07-01 12:54 +0800
Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP · conc 4 vLLM + MTP NVFP4 65536 4 232.4 done 2026-07-01 12:37 +0800
Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP · conc 2 vLLM + MTP NVFP4 65536 2 161.2 done 2026-07-01 12:20 +0800
Qwen3.6-35B-A3B · DDTree (Diffusion Draft Tree) · BLOCKED (hybrid GDN cache) DDTree research harness (github.com/liranringel/ddtree, PyTorch + transformers, batch-1) + DDTree — tree draft over the z-lab/Qwen3.6-35B-A3B-DFlash block-diffusion drafter BF16 (harness loads the unquantized target via AutoModelForCausalLM) 4096 1 — blocked 2026-07-01
Qwen3.6-27B (dense) · DDTree (Diffusion Draft Tree) · BLOCKED (hybrid GDN cache) DDTree research harness (github.com/liranringel/ddtree, PyTorch + transformers, batch-1) + DDTree — tree draft over the z-lab/Qwen3.6-27B-DFlash block-diffusion drafter BF16 (harness loads the unquantized target via AutoModelForCausalLM) 4096 1 — blocked 2026-07-01
Qwen3.6-35B-A3B-heretic (AEON) · vLLM-ultimate · NVFP4 + DFlash · conc 1/8/32 vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash @31977fbe small-page rev, num_speculative_tokens 11) NVFP4 (compressed-tensors, nvfp4-pack-quantized — GDN gates kept in BF16) 40960 1 77.6 done 2026-06-28 20:52 +0800
Ornith-1.0-35B AEON Uncensored · vLLM (aeon-ultimate) · NVFP4 + DFlash · conc 1 — BLOCKED vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash, num_speculative_tokens 6) — BLOCKED at KV unification NVFP4 (compressed-tensors, nvfp4-pack-quantized) 262144 1 — blocked 2026-06-28 16:13 +0800
Ornith-1.0-35B AEON Uncensored · vLLM (aeon-ultimate) · NVFP4 · conc 32 vLLM (aeon-vllm-ultimate custom container) + none (DFlash OFF — see the dflash-blocked page; DFlash trips hybrid+draft KV unification) NVFP4 (compressed-tensors, nvfp4-pack-quantized) 32768 32 422.1 done 2026-06-28 16:13 +0800
Ornith-1.0-35B AEON Uncensored · vLLM (aeon-ultimate) · NVFP4 · conc 1 (256K ctx) vLLM (aeon-vllm-ultimate custom container) + none (DFlash OFF — DFlash trips hybrid+draft KV unification, see dflash-blocked page) NVFP4 (compressed-tensors, nvfp4-pack-quantized) 262144 1 37.7 done 2026-06-28 16:13 +0800
Qwen3.6-27B AEON Uncensored · vLLM-ultimate (custom) · NVFP4 + DFlash · conc 32 vLLM (aeon-vllm-ultimate custom container) + DFlash (z-lab drafter, num_speculative_tokens 12) NVFP4 (XS mixed-precision) 65536 32 184.2 done 2026-06-24 01:35 +0800
Qwen3.6-27B AEON Uncensored · vLLM-ultimate (custom) · NVFP4 + DFlash · conc 8 vLLM (aeon-vllm-ultimate custom container) + DFlash (z-lab drafter, num_speculative_tokens 12) NVFP4 (XS mixed-precision) 65536 8 127.1 done 2026-06-24 01:11 +0800
Qwen3.6-27B AEON Uncensored · vLLM-ultimate (custom) · NVFP4 + DFlash · conc 1 vLLM (aeon-vllm-ultimate custom container) + DFlash (z-lab drafter, num_speculative_tokens 12) NVFP4 (XS mixed-precision) 65536 1 29.9 done 2026-06-24 00:47 +0800
Qwen3.6-27B AEON Uncensored · vLLM · NVFP4 + MTP (XS) · conc 1 vLLM + MTP (qwen3_5_mtp, grafted) NVFP4 (XS mixed-precision) 65536 1 19.6 done 2026-06-23 14:24 +08
Qwen3.6-27B AEON Uncensored · vLLM · NVFP4 + MTP (XS) · conc 8 vLLM + MTP (qwen3_5_mtp, grafted) NVFP4 (XS mixed-precision) 65536 8 127.5 done 2026-06-23 14:11 +08
Qwen3.6-27B AEON Uncensored · vLLM · NVFP4 + MTP (XS) vLLM + MTP (qwen3_5_mtp, grafted) NVFP4 (XS mixed-precision) 65536 32 303.3 done 2026-06-23 13:56 +08
Qwen3.6-35B-A3B · SGLang · NVFP4 SGLang NVFP4 65536 32 — blocked 2026-06-23 13:08 +08
Qwen3.6-35B-A3B · SGLang · NVFP4 + MTP SGLang + MTP (NEXTN) NVFP4 65536 32 — blocked 2026-06-23 13:08 +08
Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP · conc 1 vLLM + MTP NVFP4 65536 1 93.9 done 2026-06-23 13:02 +08
Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP · conc 8 vLLM + MTP NVFP4 65536 8 289.1 done 2026-06-23 12:49 +08
Qwen3.6-35B-A3B · vLLM · NVFP4 + MTP vLLM + MTP NVFP4 65536 32 541.3 done 2026-06-23 12:34 +08
Qwen3.6-27B · SGLang · NVFP4 + MTP SGLang + MTP (NEXTN) NVFP4 65536 32 196.6 done 2026-06-23 12:18 +08
Qwen3.6-27B · SGLang · NVFP4 SGLang NVFP4 65536 32 178.0 done 2026-06-23 11:35 +08
Qwen3.6-35B-A3B · vLLM · NVFP4 vLLM NVFP4 65536 32 430.8 done 2026-06-23 11:13 +08
Qwen3.6-27B · vLLM · NVFP4 + MTP · conc 1 vLLM + MTP NVFP4 65536 1 16.9 done 2026-06-23 10:48 +08
Qwen3.6-27B · vLLM · NVFP4 + MTP · conc 8 vLLM + MTP NVFP4 65536 8 109.1 done 2026-06-23 10:33 +08
Qwen3.6-27B · vLLM · NVFP4 + MTP vLLM + MTP NVFP4 65536 32 274.1 done 2026-06-23 10:19 +08
Qwen3.6-27B · vLLM · NVFP4 vLLM NVFP4 65536 32 187.7 done 2026-06-23 09:54 +08
Gemma 4 31B · vLLM · NVFP4 + EAGLE3 · conc 8 vLLM + EAGLE3 NVFP4 65536 8 91.5 done 2026-06-23 01:13 +08
Gemma 4 31B · vLLM · NVFP4 + EAGLE3 · conc 1 vLLM + EAGLE3 NVFP4 65536 1 14.8 done 2026-06-23 01:01 +08
Gemma 4 31B · llama.cpp · Q4_K_M + MTP · conc 8 llama.cpp + MTP (Google assistant drafter) Q4_K_M 65536 8 62.2 done 2026-06-23 00:24 +08
Gemma 4 31B · llama.cpp · Q4_K_M + MTP · conc 1 llama.cpp + MTP (Google assistant drafter) Q4_K_M 65536 1 24.0 done 2026-06-23 00:18 +08
gpt-oss-20b · vLLM · MXFP4 + EAGLE3 · conc 8 vLLM + EAGLE3 MXFP4 65536 8 126.3 done 2026-06-22 23:13 +08
gpt-oss-20b · vLLM · MXFP4 + EAGLE3 · conc 1 vLLM + EAGLE3 MXFP4 65536 1 38.6 done 2026-06-22 23:03 +08
Qwen3.6-35B-A3B · vLLM · FP8 + MTP · conc 1 vLLM + MTP FP8 65536 1 54.0 done 2026-06-22 20:24 +08
Qwen3.6-35B-A3B · vLLM · FP8 + MTP · conc 8 vLLM + MTP FP8 65536 8 190.3 done 2026-06-22 20:11 +08
Qwen3.6-27B · vLLM · FP8 + MTP · conc 1 vLLM + MTP FP8 65536 1 15.2 done 2026-06-22 19:55 +08
Qwen3.6-27B · vLLM · FP8 + MTP · conc 8 vLLM + MTP FP8 65536 8 98.0 done 2026-06-22 19:42 +08
Qwen3.6-35B-A3B · vLLM · FP8 + MTP vLLM + MTP FP8 65536 32 407.9 done 2026-06-22 19:28 +08
Qwen3.6-35B-A3B · vLLM · FP8 vLLM FP8 65536 32 286.0 done 2026-06-22 19:09 +08
Qwen3.6-27B · vLLM · FP8 + MTP vLLM + MTP FP8 65536 32 240.9 done 2026-06-22 18:46 +08
Qwen3.6-27B · vLLM · FP8 vLLM FP8 65536 32 154.7 done 2026-06-22 18:11 +08
gpt-oss-20b · vLLM · MXFP4 + EAGLE3 vLLM + EAGLE3 MXFP4 65536 32 686.5 done 2026-06-22 17:52 +08
Gemma 4 31B · llama.cpp · Q4_K_M + MTP llama.cpp + MTP (Google assistant drafter) Q4_K_M 65536 32 93.0 done 2026-06-22 11:36 +08
Gemma 4 31B · vLLM · NVFP4 + EAGLE3 vLLM + EAGLE3 NVFP4 65536 32 264.7 done 2026-06-22 10:51 +08
Gemma 4 31B · vLLM · FP8 vLLM FP8 65536 32 147.6 done 2026-06-22 10:10 +08
Gemma 4 26B-A4B · vLLM · FP8 vLLM FP8 65536 32 316.9 done 2026-06-22 09:47 +08
Gemma 4 31B · vLLM · NVFP4 vLLM NVFP4 65536 32 167.0 done 2026-06-22 09:12 +08
DeepSeek-R1-Distill-Qwen-32B · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 117.6 done 2026-06-22 08:49 +08
Gemma 4 31B · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 78.5 done 2026-06-22 08:29 +08
Gemma 4 26B-A4B · vLLM · BF16 vLLM BF16 65536 32 190.1 done 2026-06-22 08:10 +08
Granite 4.1 30B · vLLM · FP8 vLLM FP8 65536 32 181.8 done 2026-06-22 07:00 +08
Qwen3-Coder-30B-A3B · vLLM · FP8 vLLM FP8 65536 32 295.8 done 2026-06-22 06:39 +08
gpt-oss-20b · vLLM · MXFP4 vLLM MXFP4 65536 32 535.3 done 2026-06-22 06:00 +08
DeepSeek-Coder-V2-Lite-Instruct · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 130.5 done 2026-06-22 05:43 +08
Nemotron-Terminal-32B · vLLM · FP8 vLLM FP8 40960 32 166.2 done 2026-06-22 01:28 +08
Nemotron-3 Nano-Omni 30B-A3B · vLLM · NVFP4 vLLM NVFP4 65536 32 388.9 done 2026-06-21 23:43 +08
Nemotron-3 Elastic 30B-A3B · vLLM · NVFP4 vLLM NVFP4 65536 32 353.0 done 2026-06-21 23:23 +08
gpt-oss-20b · SGLang · MXFP4 SGLang MXFP4 65536 32 279.5 done 2026-06-21 22:08 +08
GLM-4.7-Flash · vLLM · FP8 vLLM FP8 131072 32 — blocked —
Granite-switch 4.1 30B · vLLM · FP8 vLLM FP8 131072 32 — blocked —
Ornith-1.0-35B AEON Uncensored · vLLM (aeon-ultimate) · NVFP4 + DFlash · conc 1 — BLOCKED vLLM (aeon-vllm-ultimate custom container, v0.23.0+aeon.sm121a.dflash) + DFlash (z-lab/Qwen3.6-35B-A3B-DFlash, num_speculative_tokens 6) — BLOCKED at KV unification NVFP4 (compressed-tensors, nvfp4-pack-quantized) 262144 1 — blocked 2026-06-28 16:13 +0800
Qwen3.6-27B (dense) · DDTree (Diffusion Draft Tree) · BLOCKED (hybrid GDN cache) DDTree research harness (github.com/liranringel/ddtree, PyTorch + transformers, batch-1) + DDTree — tree draft over the z-lab/Qwen3.6-27B-DFlash block-diffusion drafter BF16 (harness loads the unquantized target via AutoModelForCausalLM) 4096 1 — blocked 2026-07-01
Qwen3.6-35B-A3B · DDTree (Diffusion Draft Tree) · BLOCKED (hybrid GDN cache) DDTree research harness (github.com/liranringel/ddtree, PyTorch + transformers, batch-1) + DDTree — tree draft over the z-lab/Qwen3.6-35B-A3B-DFlash block-diffusion drafter BF16 (harness loads the unquantized target via AutoModelForCausalLM) 4096 1 — blocked 2026-07-01
Qwen3.6-35B-A3B · SGLang · NVFP4 SGLang NVFP4 65536 32 — blocked 2026-06-23 13:08 +08
Qwen3.6-35B-A3B · SGLang · NVFP4 + MTP SGLang + MTP (NEXTN) NVFP4 65536 32 — blocked 2026-06-23 13:08 +08
Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-16 SGLang + DFlash2 NVFP4 262144 16 — blocked 2026-08-20 05:25 +0800
Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-32 SGLang + DFlash2 NVFP4 262144 32 — blocked 2026-08-20 05:25 +0800
Qwen3.8-27B · SGLang · NVFP4 + DFlash2 · conc-8 SGLang + DFlash2 NVFP4 262144 8 — blocked 2026-08-20 05:25 +0800
Qwen3.8-27B · SGLang · NVFP4 + DSpark · conc-16 SGLang + DSpark NVFP4 262144 16 — blocked 2026-08-20 05:25 +0800
Qwen3.8-27B · SGLang · NVFP4 + DSpark · conc-32 SGLang + DSpark NVFP4 262144 32 — blocked 2026-08-20 05:25 +0800
gpt-oss-20b · llama.cpp · MXFP4 llama.cpp MXFP4 131072 32 — blocked 2026-06-21

41-130B (23)

ConfigurationEngineQuantCtxConcDecode tok/sStatusCompleted
Nemotron-3 Puzzle 75B-A9B · vLLM · NVFP4 · conc 32 vLLM NVFP4 65536 32 177.8 done 2026-07-11 17:21 +08
Nemotron-3 Puzzle 75B-A9B · vLLM · NVFP4 · conc 8 vLLM NVFP4 65536 8 94.0 done 2026-07-11 16:54 +08
Nemotron-3 Puzzle 75B-A9B · vLLM · NVFP4 · conc 1 vLLM NVFP4 65536 1 19.8 done 2026-07-11 16:29 +08
gpt-oss-120b · vLLM · MXFP4 + EAGLE3 (LMSYS draft) · conc 8 vLLM + EAGLE3 (lmsys/EAGLE3-gpt-oss-120b-bf16 — the SGLang/SpecForge draft, on vLLM) MXFP4 65536 8 175.3 done 2026-07-01 19:40 +0800
gpt-oss-120b · vLLM · MXFP4 + EAGLE3 (LMSYS draft) · conc 32 vLLM + EAGLE3 (lmsys/EAGLE3-gpt-oss-120b-bf16 — the SGLang/SpecForge draft, on vLLM) MXFP4 65536 32 246.7 done 2026-07-01 19:21 +0800
gpt-oss-120b · vLLM · MXFP4 + EAGLE3 (LMSYS draft) · conc 1 vLLM + EAGLE3 (lmsys/EAGLE3-gpt-oss-120b-bf16 — the SGLang/SpecForge draft, on vLLM) MXFP4 65536 1 25.5 done 2026-07-01 17:37 +0800
GLM-4.5-Air-REAP 82B · vLLM · NVFP4 (W4A16) vLLM NVFP4 (W4A16) 65536 32 158.4 done 2026-06-26 16:10 +0800
Qwen3.5-122B-A10B · vLLM · int4-AutoRound-EC + DFlash · conc 8 vLLM + DFlash (z-lab drafter, rev 6c7242c pinned) int4-AutoRound-EC 262144 8 107.4 done 2026-06-24 08:07 +0800
gpt-oss-120b · SGLang · MXFP4 + EAGLE3 · conc 32 SGLang + EAGLE3 MXFP4 65536 32 171.9 done 2026-06-23 21:20 +0800
gpt-oss-120b · SGLang · MXFP4 + EAGLE3 · conc 1 SGLang + EAGLE3 MXFP4 65536 1 40.6 done 2026-06-23 20:08 +0800
Qwen3.5-122B-A10B · vLLM · int4-AutoRound-EC · conc 8 vLLM int4-AutoRound-EC 262144 8 85.5 done 2026-06-23 14:54 +08
gpt-oss-120b · vLLM · MXFP4 + EAGLE3 · conc 8 vLLM + EAGLE3 MXFP4 65536 8 50.0 done 2026-06-22 23:44 +08
gpt-oss-120b · vLLM · MXFP4 + EAGLE3 · conc 1 vLLM + EAGLE3 MXFP4 65536 1 14.7 done 2026-06-22 23:29 +08
gpt-oss-120b · vLLM · MXFP4 + EAGLE3 vLLM + EAGLE3 MXFP4 65536 32 138.5 done 2026-06-22 17:42 +08
Llama 4 Scout 17B-16E · vLLM · AWQ-Int4 vLLM W4A16 65536 32 61.5 done 2026-06-22 17:15 +08
Qwen3-Coder-Next · vLLM · FP8 vLLM FP8 262144 32 184.7 done 2026-06-22 16:54 +08
DeepSeek-R1-Distill-Llama-70B · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 52.8 done 2026-06-22 16:30 +08
Llama 3.3 70B · llama.cpp · Q4_K_M llama.cpp Q4_K_M 65536 32 48.7 done 2026-06-22 16:01 +08
gpt-oss-120b · vLLM · MXFP4 vLLM MXFP4 65536 32 252.8 done 2026-06-22 15:30 +08
Mistral Small 4 119B · vLLM · NVFP4 vLLM NVFP4 65536 32 135.0 done 2026-06-22 02:50 +08
Nemotron-3 Super 120B · vLLM · NVFP4 vLLM NVFP4 65536 32 96.9 done 2026-06-22 00:22 +08
gpt-oss-120b · SGLang · MXFP4 SGLang MXFP4 65536 32 140.3 done 2026-06-21 22:56 +08
GLM-4.5-Air · vLLM · (quant TBD) vLLM TBD 65536 32 — blocked —

130B+ (12)

ConfigurationEngineQuantCtxConcDecode tok/sStatusCompleted
DeepSeek V4-Flash REAP-K180 · ds4 · NVFP4-hybrid · conc 32 (32K ctx) ds4 (ds4-nvfp4-spark, custom GB10 runtime) + none — ds4 is a serial single-worker; the DeepSeek MTP head is not exercised by this runtime NVFP4-hybrid (NVFP4 experts + Q2_K + Q8_0 + F16) 32768 32 11.0 done 2026-06-28 13:34 +08
DeepSeek V4-Flash REAP-K180 · ds4 · NVFP4-hybrid · conc 1 (1M ctx) ds4 (ds4-nvfp4-spark, custom GB10 runtime) + none — ds4 is a serial single-worker; the DeepSeek MTP head is not exercised by this runtime NVFP4-hybrid (NVFP4 experts + Q2_K + Q8_0 + F16) 1048576 1 11.4 done 2026-06-28 13:34 +08
MiniMax-M2.7-REAP 172B · vLLM · NVFP4 (W4A4) vLLM NVFP4 (W4A4) 65536 32 111.9 done 2026-06-27 19:22 +0800
MiniMax-M2.7-REAP 172B · vLLM · NVFP4 (W4A4) · conc 1 · 160K vLLM NVFP4 (W4A4) 163840 1 25.4 done 2026-06-27 19:22 +0800
MiniMax-M2.7 · llama.cpp · UD-Q3_K_XL llama.cpp UD-Q3_K_XL 65536 32 58.8 done 2026-06-24 06:30 +0800
MiniMax-M2.7 · llama.cpp · UD-IQ4_XS llama.cpp UD-IQ4_XS 32768 32 58.4 done 2026-06-23 22:37 +0800
DeepSeek V4-Flash · llama.cpp · IQ2_XXS-XL (GGUF) llama.cpp IQ2_XXS-XL 8192 8 — blocked 2026-06-23
DeepSeek V4-Flash · vLLM · AWQ-Int4 vLLM NVFP4 131072 32 — blocked —
DeepSeek V4-Flash · vLLM · NVFP4 + EAGLE3.1 vLLM + EAGLE3.1 NVFP4 65536 32 — blocked —
Llama 4 Maverick 402B · llama.cpp · UD-Q2_K_XL llama.cpp UD-Q2_K_XL 131072 32 — blocked —
MiniMax-M2.7 · llama.cpp · MXFP4_MOE llama.cpp MXFP4_MOE 65536 32 — blocked —
MiniMax-M2.7 · vLLM · NVFP4 vLLM NVFP4 65536 32 — blocked —