Projects / TurboQuant KV Cache Optimization / Experiments / Temporal decay — progressive 3-to-2 bit requantization

Temporal decay — progressive 3-to-2 bit requantization

proposed medium priority TODO-010
Overview Experiments 96 Forks 3 Resources 36 Benchmarks 2 Broadcasts 3 Related
Description

Old tokens requantized turbo3_tcq to turbo2_tcq. ~30% extra memory savings at acceptable quality cost for tokens with negligible attention weight

Suggested Parameters
decay_threshold_positions 16384
source turbo3_tcq
target turbo2_tcq
Provenance
Proposed by @buun via cuda-rtx3090 claude-opus-4-6