Granite-switch 4.1 30B · vLLM · FP8
status: blocked
Configuration
| Model | ibm-granite/granite-switch-4.1-30b-preview |
|---|---|
| Company | IBM |
| Family | Granite |
| Parameters | 32B (MoE) |
| Engine | vLLM |
| Quant / precision | FP8 |
| Why this quant | Near-BF16 quality at half the bytes; official FP8 weights published. |
| Download | ibm-granite/granite-switch-4.1-30b-preview |
| Context window | 131072 |
| Input modalities | text |
Measured results
| Prefill tok/s | — |
|---|---|
| Decode tok/s | — |
| Peak memory (GB) | — |
Full run command
# blocked — no confirmable repo (see Notes)
Notes
Blocked — no confirmable “switch” Granite repo. Verified 2026-06-22.
ibm-granite/granite-switch-4.1-30b-preview→ 404 on HF; no IBM “switch”-named MoE resolves.- The real ~30B Granite MoE is
ibm-granite/granite-4.1-30b— already benchmarked as its own config (granite-4-1-30b-vllm-fp8, 221/182 tok/s). There is alsoibm-granite/granite-4.0-h-small(a hybrid), but neither is a distinct “switch” architecture, so running one here would just duplicate an existing config under a wrong name. - Per “when unsure, BLOCK — don’t guess,” left for human review. If IBM ships a genuine
switch-MoE Granite, point
source_repoat it; otherwise this config is a duplicate of granite-4.1-30b and can be dropped.