Qun Liu, Pengfei Li, Meng Zhang, Liangyou Li, Minghao Wu
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Pre-trained sequence-to-sequence models have significantly improved Neural Machine Translation (NMT). Different from prior works where pre-trained models usually adopt an unidirectional decoder, this paper demonstrates that pre-training a sequenceto-sequence model but with a bidirectional decoder can produce notable performance gains for both Autoregressive and Nonautoregressive NMT. Specifically, we propose CeMAT, a conditional masked language model pre-trained on large-scale bilingual and monolingual corpora in many languages. 1 We also introduce two simple but effective methods to enhance the CeMAT, aligned code-switching & masking and dynamic dual-masking. We conduct extensive experiments and show that our CeMAT can achieve significant performance improvement for all scenarios from low-to extremely highresource languages, i.e., up to +14.4 BLEU on low-resource and +7.9 BLEU on average for Autoregressive NMT. For Non-autoregressive NMT, we demonstrate it can also produce consistent performance gains, i.e., up to +5.3 BLEU. To the best of our knowledge, this is the first work to pre-train a unified model for fine-tuning on both NMT tasks.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2203.09210")
get_code_for_paper("2203.09210")
have("2203.09210")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2203.09210.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2203.09210)
Connect an agent — have() is free.