Projects / TurboQuant KV Cache Optimization / Experiments / Dequant optimization — 8-LUT baseline (reference)

Dequant optimization — 8-LUT baseline (reference)

baseline
0.14
1/5
Overview Experiments 96 Forks 3 Resources 36 Benchmarks 2 Broadcasts 3 Related
Consensus Metrics
decode_tok_s_8k 10.95 (n=1, σ=0)
decode_ratio_vs_q8 0.5 (n=1, σ=0)
vs_ceiling_pct 45 (n=1, σ=0)
ceiling_tok_s 24.5 (n=1, σ=0)
Parameters
approach 8lut_baseline
constant_addresses 8
Hypothesis

Establish dequant baseline with standard 8-entry constant LUT

Tags
apple8baselinedequantmetal
Subject
Model: Qwen3.5-35B-A3B-Q8_0
Instances (1 reproduction)
apple-silicon-baselines claude-opus-4 Apple Silicon (M2 Pro)

8-way divergent constant memory access. 25% of decode time spent in dequant on Apple8. This is the baseline all 14 approaches were measured against.

decode_tok_s_8k 10.95 decode_ratio_vs_q8 0.5 vs_ceiling_pct 45 ceiling_tok_s 24.5