Lijun Wu, Tao Qin, Tie-Yan Liu, Juntao Li, Min Zhang, Yisheng Xiao, Ruiyang Xu
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Transformer-based autoregressive (AR) methods have achieved appealing performance for varied sequence-tosequence generation tasks, e.g., neural machine translation, summarization, and code generation, but suffer from low inference efficiency. To speed up the inference stage, many non-autoregressive (NAR) strategies have been proposed in the past few years. Among them, the conditional masked language model (CMLM) is one of the most versatile frameworks, as it can support many different sequence generation scenarios and achieve very competitive performance on these tasks. In this paper, we further introduce a simple yet effective adaptive masking over masking strategy to enhance the refinement capability of the decoder and make the encoder optimization easier. Experiments on 3 different tasks (neural machine translation, summarization, and code generation) with 15 datasets in total confirm that our proposed simple method achieves significant performance improvement over the strong CMLM model. Surprisingly, our proposed model yields state-of-the-art performance on neural machine translation (34.62 BLEU on WMT16 EN→RO, 34.82 BLEU on WMT16 RO→EN, and 34.84 BLEU on IWSLT De→En) and even better performance than the AR Transformer on 7 benchmark datasets with at least 2.2× speedup. Our code is available at GitHub 1 .
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2303.07457")
get_code_for_paper("2303.07457")
have("2303.07457")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2303.07457.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2303.07457)
Connect an agent — have() is free.