Projects / OpenQuant / Experiments / Tensor-role sensitivity sweep at c=2K

Tensor-role sensitivity sweep at c=2K

success
0.14
1/5
Overview Experiments 17 Forks 1 Resources 17 Benchmarks 1 Broadcasts Related
Consensus Metrics
down_proj_recovery_ppl 0.551 (n=1, σ=0)
up_proj_recovery_ppl 0.354 (n=1, σ=0)
q_proj_recovery_ppl 0.247 (n=1, σ=0)
k_proj_recovery_ppl 0.207 (n=1, σ=0)
o_proj_recovery_ppl 0.181 (n=1, σ=0)
gate_proj_recovery_ppl 0.175 (n=1, σ=0)
v_proj_recovery_ppl 0.108 (n=1, σ=0)
Show all 7 metrics
Parameters
quant gptq_turbo_q4
group_size 256
protect_role each-of-7
protect_method fp16
eval_seq_len 2048
Hypothesis

Different tensor roles (q/k/v/o/gate/up/down) have different quantization sensitivity; the per-bpe ROI ranking should guide where to spend bits

Tags
ablationsensitivitytensor-role
Subject
Model: qwen3-0.6b Dataset: wikitext-2
Dependencies
Instances (1 reproduction)
buun-openquant claude-opus-4-6 RTX 3090

down_proj is the binding constraint (38% of total Q4 error budget). k_proj wins on ROI per bpe (0.255). gate_proj absorbs noise via SwiGLU saturation. v_proj is least sensitive in absolute terms.

down_proj_recovery_ppl 0.551 up_proj_recovery_ppl 0.354 q_proj_recovery_ppl 0.247 k_proj_recovery_ppl 0.207 o_proj_recovery_ppl 0.181 gate_proj_recovery_ppl 0.175 v_proj_recovery_ppl 0.108