Granite-switch 4.1 30B · vLLM · FP8

status: blocked

Configuration

Modelibm-granite/granite-switch-4.1-30b-preview
CompanyIBM
FamilyGranite
Parameters32B (MoE)
EnginevLLM
Quant / precisionFP8
Why this quantNear-BF16 quality at half the bytes; official FP8 weights published.
Downloadibm-granite/granite-switch-4.1-30b-preview
Context window131072
Input modalitiestext

Measured results

Prefill tok/s
Decode tok/s
Peak memory (GB)

Full run command

# blocked — no confirmable repo (see Notes)

granite-switch-4.1-30b-preview IBM Granite FP8 16-40B conc-32

Notes

Blocked — no confirmable “switch” Granite repo. Verified 2026-06-22.

  • ibm-granite/granite-switch-4.1-30b-preview404 on HF; no IBM “switch”-named MoE resolves.
  • The real ~30B Granite MoE is ibm-granite/granite-4.1-30b — already benchmarked as its own config (granite-4-1-30b-vllm-fp8, 221/182 tok/s). There is also ibm-granite/granite-4.0-h-small (a hybrid), but neither is a distinct “switch” architecture, so running one here would just duplicate an existing config under a wrong name.
  • Per “when unsure, BLOCK — don’t guess,” left for human review. If IBM ships a genuine switch-MoE Granite, point source_repo at it; otherwise this config is a duplicate of granite-4.1-30b and can be dropped.