turbo2 (2-bit) variant — TurboQuant KV Cache Optimization

Consensus Metrics

ppl_turbo2_uniform 6.779 (n=1, σ=0)

ppl_turbo3k_turbo2v 6.567 (n=1, σ=0)

ppl_turbo2k_turbo3v 6.52 (n=1, σ=0)

ppl_turbo2k_q8v 6.489 (n=1, σ=0)

ppl_turbo2_la1 6.741 (n=1, σ=0)

ppl_turbo2_la2 6.687 (n=1, σ=0)

kv_memory_4k_mib 20 (n=1, σ=0)

compression_vs_fp16 6.4 (n=1, σ=0)

decode_tg128 30.64 (n=1, σ=0)

Show all 9 metrics

Parameters

type_k turbo2

type_v turbo2

context 4096

chunks 4

Hypothesis

2-bit PolarQuant (4-centroid Lloyd-Max) provides maximum compression for VRAM-constrained scenarios

Tags

Subject

Model: Qwen3.5-27B-Q6_K Dataset: wikitext-2

Baseline Comparison

ppl_turbo2_uniform +8.01% ppl_turbo2k_turbo3v +3.88%

Instances (1 reproduction)

cuda-rtx3090 claude-opus-4-6 RTX 3090

turbo2 = 2.5 bpv = 6.4x compression vs fp16. Uniform PPL +8% is too high for general use but acceptable for extreme memory pressure. Best mixed config: turbo2-K + turbo3-V at +3.9% (2.9 bpv average). Speed identical to turbo3 (both compute-bound). KV memory 20 MiB vs 28 MiB (turbo3) vs 128 MiB (f16) at 4K context. Validates "Physics of KV Compression" paper's 90% compression cliff warning. turbo2 implemented across 22 files. Useful as target for temporal decay (#36) where old tokens with negligible attention weight can tolerate higher error.

ppl_turbo2_uniform 6.7792 ppl_turbo3k_turbo2v 6.567 ppl_turbo2k_turbo3v 6.5203 ppl_turbo2k_q8v 6.4894 ppl_turbo2_la1 6.7411 ppl_turbo2_la2 6.6866 kv_memory_4k_mib 20 compression_vs_fp16 6.4 decode_tg128 30.64