Tao Ge, Furu Wei, Heming Xia, Peiyi Wang, Si-Qing Chen, Zhifang Sui
We lifted 5 functions out of this paper's own repositories and ran 5 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| hemingkx/specdec | canonical | 1 of 1 |
| copy not recorded | — | 2 of 2 |
| lucidrains/speculative-decoding | extension | 1 of 1 |
| romsto/Speculative-Decoding | reimplementation | 1 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| log | Ran | lucidrains/speculative-decoding/speculative_decoding/speculative_decoding.py code served (permissive licence) · get_code("b5472673332de78e") |
| baseline_forward_decoder | Ran | hemingkx/specdec/inference.py pointer only (licence: NONE) · get_code("061c2f76d8c5307c") |
| default | Ran | this paper's copy was not recorded; identical code first harvested from ThomasMrY/VCT pointer only · get_code("60fff7c3c400d7ff") |
| exists | Ran | this paper's copy was not recorded; identical code first harvested from ThomasMrY/VCT pointer only · get_code("aa5486a3650902d8") |
| max_fn | Ran | romsto/Speculative-Decoding/sampling/speculative_decoding.py code served (permissive licence) · get_code("c2abbdc78126fdbd") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We propose Speculative Decoding (SpecDec), for the first time ever 1 , to formally study exploiting the idea of speculative execution to accelerate autoregressive (AR) decoding. Speculative Decoding has two innovations: Spec-Drafter -an independent model specially optimized for efficient and accurate drafting -and Spec-Verification -a reliable method for verifying the drafted tokens efficiently in the decoding paradigm. Experimental results on various seq2seq tasks including machine translation and abstractive summarization show our approach can achieve around 5× speedup for the popular Transformer architectures with comparable generation quality to beam search decoding, refreshing the impression that the draft-then-verify paradigm introduces only 1.4×∼2× speedup. In addition to the remarkable speedup, we also demonstrate 3 additional advantages of SpecDec, revealing its practical value for accelerating generative models in real-world applications. Our models and codes are available at https://github.com/ hemingkx/SpecDec.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2203.16487")
get_code_for_paper("2203.16487")
have("2203.16487")
Connect an agent — have() is free.