We lifted 2 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| Entropy-xcy/bitnet158 | reimplementation | 1 of 1 |
| microsoft/bitblas | pwc_unofficial | 1 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| absmax_quantize | Ran | Entropy-xcy/bitnet158/bitnet158/bitlinear158.py pointer only (licence: AFL-3.0) · get_code("2346bd33c6ee4a6a") |
| get_rasterization_code | Ran | microsoft/bitblas/bitblas/base/utils.py code served (permissive licence) · get_code("bb6778a281ed0015") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Recent research, such as BitNet, is paving the way for a new era of 1-bit Large Language Models (LLMs). In this work, we introduce a 1-bit LLM variant, namely BitNet b1.58, in which every single parameter (or weight) of the LLM is ternary {-1, 0, 1}. It matches the full-precision (i.e., FP16 or BF16) Transformer LLM with the same model size and training tokens in terms of both perplexity and end-task performance, while being significantly more cost-effective in terms of latency, memory, throughput, and energy consumption. More profoundly, the 1.58-bit LLM defines a new scaling law and recipe for training new generations of LLMs that are both high-performance and cost-effective. Furthermore, it enables a new computation paradigm and opens the door for designing specific hardware optimized for 1-bit LLMs.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2402.17764")
get_code_for_paper("2402.17764")
have("2402.17764")
Connect an agent — have() is free.