Gemma-3 SWA V cache bug fix — TurboQuant KV Cache Optimization

Consensus Metrics

ppl_q8_baseline 5.699 (n=1, σ=0)

ppl_turbo3_kv_after_fix 5.887 (n=1, σ=0)

ppl_turbo3_k_only 5.963 (n=1, σ=0)

ppl_turbo3_kv_before_fix 4.5e+13 (n=1, σ=0)

Parameters

type_k turbo3

type_v turbo3

context 2048

chunks 8

Hypothesis

Gemma-3 V cache broken because V un-rotation missing from iSWA build_attn overload

Tags

Subject

Model: Gemma-3-27B-it-Q4_K_M Dataset: wikitext-2

Baseline Comparison

ppl_turbo3_kv +3.3%

Instances (1 reproduction)

cuda-rtx3090 claude-opus-4-6 RTX 3090

V inverse rotation (ggml_turbo_wht) was missing from iSWA build_attn overload in llama-graph.cpp. Gemma-3 uses iSWA for ALL layers, so every V was un-rotated garbage (PPL 45 trillion). Fix: added V un-rotation block to iSWA build_attn after build_attn_mha, before W_O. Post-fix, Gemma-3 turbo3 K+V PPL 5.8867 (+3.3%) matches the head_dim=128 degradation pattern seen on MN-Violet-Lotus (+2.6%) and Qwen3-14B (+3.8%). K-only slightly worse than K+V, consistent with "V matters more" finding.

ppl_q8_baseline 5.6995 ppl_turbo3_kv_after_fix 5.8867 ppl_turbo3_k_only 5.9633 ppl_turbo3_kv_before_fix 45000000000000