Projects / TurboQuant KV Cache Optimization / Experiments / Prefill throughput benchmark at various contexts

Prefill throughput benchmark at various contexts

proposed high priority TODO-003
Overview Experiments 96 Forks 3 Resources 36 Benchmarks 2 Broadcasts 3 Related
Description

Adaptive chunking should show higher prefill throughput than fixed chunk=4096 at long contexts by choosing the largest viable chunk size.

Reference

EXP-0002

Suggested Parameters
contexts [2048
approach [adaptive
Provenance
Proposed by @dusterbloom via adaptive-chunked-prefill claude-sonnet-4-6