Dacheng Tao, Liang Ding, Longyue Wang, Xuebo Liu, Derek Wong, Zhaopeng Tu
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Knowledge distillation (KD) is commonly used to construct synthetic data for training non-autoregressive translation (NAT) models. However, there exists a discrepancy on lowfrequency words between the distilled and the original data, leading to more errors on predicting low-frequency words. To alleviate the problem, we directly expose the raw data into NAT by leveraging pretraining. By analyzing directed alignments, we found that KD makes low-frequency source words aligned with targets more deterministically but fails to align sufficient low-frequency words from target to source. Accordingly, we propose reverse KD to rejuvenate more alignments for lowfrequency target words. To make the most of authentic and synthetic data, we combine these complementary approaches as a new training strategy for further boosting NAT performance. We conduct experiments on five translation benchmarks over two advanced architectures. Results demonstrate that the proposed approach can significantly and universally improve translation quality by reducing translation errors on low-frequency words. Encouragingly, our approach achieves 28.2 and 33.9 BLEU points on the WMT14 English-German and WMT16 Romanian-English datasets, respectively. Our code, data, and trained models are available at https://github.com/ longyuewangdcu/RLFW-NAT.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2106.00903")
get_code_for_paper("2106.00903")
have("2106.00903")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2106.00903.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2106.00903)
Connect an agent — have() is free.