Dan Alistarh, Eldar Kurtić, Michael Helcig
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Model quantization has become essential for efficient large language model deployment, yet existing approaches present clear trade-offs: methods such as GPTQ and AWQ achieve practical compression but are lossy, while lossless techniques preserve fidelity but lack inference acceleration. This paper explores the middle ground of statistically-lossless compression, examining three complementary aspects of what losslessness means for quantized LLMs. First, task-lossless compression preserves zero-shot benchmark accuracy within natural sampling variance and is achievable at aggressive bitwidths. Second, we formalize the stricter notion of distribution-lossless compression, requiring the quantized model's next-token distribution to be practically indistinguishable from the original, and propose the Expected Acceptance Rate (EAR), the maximum token-agreement probability under optimal coupling, as a directly interpretable fidelity metric (e.g. EAR ≥ 0.99 means 99% agreement). Third, we prove a gamma-squared variance law showing that symmetric quantization inflates noise variance by γ 2 relative to asymmetric, making asymmetric quantization a prerequisite for distribution-lossless fidelity but not for task-level preservation. Through SLQ, a layer-wise non-uniform method with asymmetric quantization and wide bitwidth search, we obtain task-lossless compression at well below 4 bits per parameter (as low as 3.3 bits depending on the model), distributionlossless compression at 5-6 bits per parameter on average, and inference speedups of 1.7-3.7× compared to FP16 using optimized kernels. Source code is available at https://github.com/IST-DASLab/SLQ.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2605.02404")
get_code_for_paper("2605.02404")
have("2605.02404")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2605.02404.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2605.02404)
Connect an agent — have() is free.