Product & Environment
- Product: Aries (
aries-rb; Aries2, 2 clusters × 4 cores) - Compiler Version: qbcompiler 1.3.0 (also reproduced on 1.2.0)
- OS: Ubuntu 24.04 (host); repro verified inside the official
mobilint/qbcompiler:1.3-cuda12.8.1-ubuntu22.04image
Description
## Summary
Compiling any Llama-architecture decoder model with **`head_dim=160`** (e.g. `hidden_size=2560`, `num_attention_heads=16`) for `aries-rb` fails deterministically in the final native backend compile stage:
```
RuntimeError: …/headerview∣vh96: View channel mismatch:
in_ch=3072, out_ch=192, in_ch_cap=3072, out_ch_cap=192, ch_gcd=3, min_depth=3
```
The same model family with `head_dim=128` compiles successfully, with everything else identical. A random-initialized 2-layer Llama is sufficient to reproduce — **no checkpoint needed, ~1 minute runtime**.
## Environment
Verified on the official **`mobilint/qbcompiler:1.3-cuda12.8.1-ubuntu22.04`** image
(qbcompiler wheel 1.3.0 installed on top of its pinned defaults: Python 3.10.12,
torch 2.7.1+cu128, transformers **5.16.1**). The failure was first observed on the
1.2 image (`mobilint/qbcompiler:1.2-cuda12.8.1-ubuntu22.04`, transformers 4.57.1,
`backend=“hf”` end of July) with the real 3B model.
| Component | Version(s) where it reproduces |
|—|—|
| qbcompiler | 1.2.0 (`backend=“hf”`) and 1.3.0 (`backend=“torch”`, presets `llm` and `llm_fast`) |
| Target device | `aries-rb` (Aries2, 2 clusters × 4 cores) |
| Host | Ubuntu 24.04, aries-driver 1.13, qbruntime 1.3.2 / 1.4.0 |
Real-world models blocked by this: our in-house 3B-parameter Llama-family instruct
model (hidden 2560 / heads 16 / kv 8 → head_dim 160), and a VLM we are porting whose
text tower uses the identical geometry.
## Minimal repro
Run on qbcompiler 1.3.0 (`backend=“torch”`; ~1 minute, random weights, no checkpoint
needed). On qbcompiler 1.2.0 the equivalent failure was observed with the real 3B model
via `backend=“hf”` — same error numbers, see below.
```python
import torch
from transformers import LlamaConfig, LlamaForCausalLM
from qbcompiler import mxq_compile
def run(tag, hidden, heads, kv):
cfg = LlamaConfig(
vocab_size=1024,
hidden_size=hidden,
intermediate_size=hidden * 2,
num_hidden_layers=2,
num_attention_heads=heads,
num_key_value_heads=kv,
max_position_embeddings=256,
use_cache=False, # mirrors the real model’s config; with use_cache=True the
# 1.3.0 torch tracer errors out earlier on the cache proxy,
# which is a separate (minor) issue and not the point here
)
model = LlamaForCausalLM(cfg).to(torch.bfloat16).eval()
seqlen = 16
feed = {
“input_ids”: torch.randint(0, 1024, (1, seqlen)),
“position_ids”: torch.arange(seqlen).unsqueeze(0),
}
try:
mxq_compile(
model, target_device="aries-rb", backend="torch", config_preset="llm",
feed_dict=feed, dynamic_axes={“input_ids”: [1], “position_ids”: [1]},
use_random_calib=True, save_path=f"/workspace/repro_{tag}.mxq",
)
print(f"##### CASE {tag}: SUCCESS #####")
except Exception as e:
print(f"##### CASE {tag}: FAILED #####", type(e)._name_, str(e)[:300])
# target head geometry (hidden 2560 / 16 heads → head_dim 160):
run(“headdim160”, hidden=2560, heads=16, kv=8)
# power-of-2 control, same everything else (head_dim 128):
run(“headdim128”, hidden=2048, heads=16, kv=8)
```
### Results
| Case | head_dim | Result |
|—|—|—|
| A | **160** | FAILS: `add_3/rmsnorm/reshape/conv2d/headerview∣vh6: View channel mismatch: in_ch=3072, out_ch=192, in_ch_cap=3072, out_ch_cap=192, ch_gcd=3, min_depth=3` |
| B | **128** | **SUCCESS** (MXQ exported) |
Both runs pass parsing and quantization completely (`quantization done.`); only the native `Compiling the Quantized model (FB)` stage fails for case A.
Same signature observed end-to-end on the real 3B checkpoint (32 layers, qbcompiler 1.3.0, `backend=“torch”`, preset `llm` and also `llm_fast`):
```
add_123/rmsnorm_0/reshape/conv2d/headerview∣vh96: View channel mismatch:
in_ch=3072, out_ch=192, in_ch_cap=3072, out_ch_cap=192, ch_gcd=3, min_depth=3
```
(qbcompiler 1.2.0 / hf backend failed identically at `model.layers.31.self_attn.q_proj/headerview∣vh96`.)
## What we established / ruled out
- **Not the checkpoint, not depth, not weights:** a 2-layer random Llama reproduces with byte-identical error numbers. Only `head_dim` differs from the passing control.
- **Not the equivalent-transformation passes:** preset `llm_fast` (QK/UD/VO/SpinR1/SpinR2/OptimizeFFN all off) fails identically.
- **Where the numbers come from:** `in_ch=3072 = 16 heads × 192`; `out_ch=192` = head_dim **160 padded up to 64-channel alignment**. `ch_gcd=3 (= 192/64)` suggests the attention head-view tiling path derives an odd channel-group depth its constraint check rejects. Officially supported LLMs appear to use power-of-two head_dim (64/128), so this path may never have been exercised with `head_dim=160`.
- **“Only the last layer fails”:** every layer shares identical shapes; the failing node is always in the final-layer/graph-tail region (`layers.31` on 32 layers, `add_123` on 32 layers, `add_3` on the 2-layer repro), consistent with the backend walking the graph tail-first and aborting at the first head-view op. Middle layers are not actually “passing” — compilation stops at the first violation.
- The in-graph op is a reshape-as-conv2d head-view (`…/reshape/conv2d/headerview`), i.e. attention’s `(B*S, H*Dh) → (B, H, S, Dh)`-style expansion; the `vh96`/`vh6` suffix scales with model size.
## Ask
1. Please confirm whether `head_dim=160` is intended to be supported for `aries-rb`. If head_dim support is restricted (e.g. powers of two / multiples of 64), please document the accepted set and surface a clear error at the **config-validation** stage instead of deep in the native compiler.
2. If it is meant to work, this looks like a bug in the headerview channel-tiling logic (`ch_gcd=3, min_depth=3` check) of the closed-source backend; the 2-layer repro above should be directly usable in your regression tests.
3. If a fix already exists in a newer qbcompiler build, let us know the earliest version that contains it.
## Attachments / provenance
- The repro script above was verified end-to-end in the official
`mobilint/qbcompiler:1.3-cuda12.8.1-ubuntu22.04` image: case A (head_dim 160) fails
with the exact error quoted, case B (head_dim 128) succeeds and exports an MXQ.
- Full logs available on request (2-layer A/B repro log; 3B real-checkpoint logs for presets `llm` and `llm_fast` on qbcompiler 1.3.0; 1.2.0 hf-backend logs from our earlier internal testing).