Qwen3-Coder-30B-A3B · DDTree vs DFlash · HumanEval (coding workload) · single-stream
status: done
Configuration
| Model | Qwen/Qwen3-Coder-30B-A3B-Instruct |
|---|---|
| Company | Alibaba |
| Family | Qwen |
| Parameters | 30B / 3B (MoE, standard attention) + DFlash external drafter |
| Engine | DDTree research harness (github.com/liranringel/ddtree, PyTorch + transformers, batch-1) + DDTree tree draft (budgets 64/256) vs single-line DFlash — z-lab/Qwen3-Coder-30B-A3B-DFlash, block_size 16 (speculative decoding) |
| Quant / precision | BF16 (harness loads the unquantized target via AutoModelForCausalLM) |
| Why this quant | EXPERIMENTS.md P0 |
| Download | Qwen/Qwen3-Coder-30B-A3B-Instruct |
| Context window | 4096 |
| Input modalities | text (served text-only here) |
Measured results
| Prefill tok/s | n/a (harness reports single-stream decode + accept-len, not prefill tok/s) |
|---|---|
| Decode tok/s | 49.34 |
| Peak memory (GB) | 113.27 (system MemAvailable delta (10s sampling) — inflated by tree/KV buffers on long full-256-tok code spans + fs page cache; batch-1 working set is ~68 GB (cf. the mt-bench run)) |
| Completed | 2026-07-01 21:26 +0800 |
Full run command
# Same harness/model/drafter as the mt-bench run; dataset = humaneval (164 single-turn code problems).
scripts/bench-ddtree.sh Qwen/Qwen3-Coder-30B-A3B-Instruct z-lab/Qwen3-Coder-30B-A3B-DFlash \
humaneval 12 256 64,256 0.0 coder-humaneval
# 12 HumanEval problems, 256-tok cap (all hit it: 3072 out toks), temp 0.0, block_size 16, cpp compaction ON.
Notes
On code, block-diffusion spec-decode is a 2.7–2.8× single-stream WIN — and the tree’s edge over the single
line nearly vanishes. Same Qwen3-Coder-30B-A3B / DFlash drafter as …-ddtree,
only the workload changes (chat → code). This is the workload thesis, measured.
| method | decode tok/s | accept-len (code) | vs base | accept-len (chat) |
|---|---|---|---|---|
| baseline (autoregressive) | 17.66 | 1.00 | — | 1.00 |
| DFlash (single-line) | 47.87 | 7.96 | 2.71× | 2.25 |
| DDTree, tree-budget 64 | 49.34 | 9.74 | 2.79× | 3.22 |
| DDTree, tree-budget 256 | 41.30 | 10.50 | 2.34× | 3.69 |
- Acceptance is overwhelmingly workload-driven — thesis confirmed. Moving chat→code, single-line DFlash accept-len jumps 2.25 → 7.96 (×3.5) and DDTree-tb64 3.22 → 9.74 (×3.0). Code is templated and low-entropy, so the block-diffusion draft nails long spans the target accepts wholesale. This is exactly the §1c/§3b “acceptance is workload- not concurrency-driven” claim, now shown for the tree method specifically.
- Spec flips from a batch-1 loss (chat) to a huge win (code). On chat both spec methods were ≤1.12× (DFlash a net loss, 0.92×). On code, DFlash is 2.71× and DDTree-tb64 2.79× over autoregressive — even at batch-1, unquantized bf16, with the SDPA-forced tree verify. So whether single-stream spec-decode is worth it is dominated by the workload, not the method.
- The tree’s marginal value is inversely related to how good the single line already is. DDTree-tb64 beats single-line DFlash by only +3.1% on code (49.34 vs 47.87) versus +22% on chat. When the single line already accepts ~8-of-16 (code), there is little headroom left for the tree to capture; when it accepts only ~2.25 (chat) the tree’s extra candidate continuations matter a lot. Takeaway: DDTree earns its keep on HARD/high-entropy workloads; on easy/templated ones plain DFlash already gets most of the win.
- Budget optimum holds, sharper here. tb256 has the highest accept-len (10.50) yet is the slowest spec config (41.30, 0.84× of tb64) — with code rounds already few (324→299), the 256-node tree’s per-step verify cost dominates. tb64 is again the sweet spot.
- Memory 113 GB is higher than the mt-bench 68 GB because every code completion runs the full 256 tokens (longer sequences, larger tree/KV buffers) and MemAvailable also absorbs fs page cache; the true batch-1 working set is ~68 GB (per the chat run).
METHODOLOGY: batch-1 only (no conc 8/32); bf16 target (absolute tok/s not comparable to the NVFP4 serving
rows); harness-supported proxy target (our Qwen3.6 hybrids are blocked — 35b-a3b).
Chat companion: qwen3-coder-30b-a3b-ddtree. See notes/INCOMPATIBILITIES.md +
scripts/bench-ddtree.sh.