Projects / OpenQuant / Experiments / Q4_K_M k-quant baseline (the bar to beat)

Q4_K_M k-quant baseline (the bar to beat)

baseline
0.14
1/5
Overview Experiments 17 Forks 1 Resources 17 Benchmarks 1 Broadcasts Related
Consensus Metrics
perplexity 19.46 (n=1, σ=0)
bits_per_param 4.84 (n=1, σ=0)
Parameters
quant Q4_K_M
group_size 32
calib null
gptq false
eval_seq_len 2048
Hypothesis

Q4_K_M is the strongest production-ready 4-bit k-quant; sub-fp16 methods need to beat this on the Pareto frontier

Tags
baselinek-quantq4_k_m
Subject
Model: qwen3-0.6b Dataset: wikitext-2
Baseline Comparison
perplexity +7.45% bits_per_param -69.75%
Instances (1 reproduction)
buun-openquant claude-opus-4-6 RTX 3090

The 4-bit Pareto frontier any new method has to clear. Q5_K_M=18.64@5.50, Q6_K=18.01@6.56, Q4_0=20.69@4.50.

perplexity 19.46 bits_per_param 4.84