We lifted 6 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| huawei-noah/Pretrained-Language-Model | canonical | 1 of 1 |
| copy not recorded | — | 3 of 5 |
| Function | Status | Where it lives |
|---|---|---|
| swish | Ran | this paper's copy was not recorded; identical code first harvested from jeonsworld/ViT-pytorch pointer only · get_code("0f786c407fb1ee4c") |
| acc_and_f1 | Ran | this paper's copy was not recorded; identical code first harvested from danrsc/bert_brain_neurips_2019 pointer only · get_code("cac113ca87b9d9f3") |
| convert_examples_to_features | Ran | huawei-noah/Pretrained-Language-Model/BinaryBERT/utils_glue.py pointer only (licence: NONE) · get_code("38412a23a52f5f6e") |
| gelu | Ran | this paper's copy was not recorded; identical code first harvested from tanmay1618/text_to_sql_bert pointer only · get_code("fdc64f4c72036ae4") |
| load_tf_weights_in_bert | Not yet run | this paper's copy was not recorded; identical code first harvested from jinglong696/secformer pointer only · get_code("0453ed29d1aecc4c") |
| simple_accuracy | Not yet run | this paper's copy was not recorded; identical code first harvested from ink-usc/expl-refinement pointer only · get_code("3c241ecfe3749a6d") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
The rapid development of large pre-trained language models has greatly increased the demand for model compression techniques, among which quantization is a popular solution. In this paper, we propose BinaryBERT, which pushes BERT quantization to the limit by weight binarization. We find that a binary BERT is hard to be trained directly than a ternary counterpart due to its complex and irregular loss landscape. Therefore, we propose ternary weight splitting, which initializes BinaryBERT by equivalently splitting from a half-sized ternary network. The binary model thus inherits the good performance of the ternary one, and can be further enhanced by fine-tuning the new architecture after splitting. Empirical results show that our BinaryBERT has only a slight performance drop compared with the full-precision model while being 24x smaller, achieving the state-of-the-art compression results on the GLUE and SQuAD benchmarks.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2012.15701")
get_code_for_paper("2012.15701")
have("2012.15701")
Connect an agent — have() is free.