Han Zhang, Yu Sun, Hao Tian, Hua Wu, Dongling Xiao, Haifeng Wang, Yukun Li
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Current pre-training works in natural language generation pay little attention to the problem of exposure bias on downstream tasks. To address this issue, we propose an enhanced multi-flow sequence to sequence pre-training and fine-tuning framework named ERNIE-GEN, which bridges the discrepancy between training and inference with an infilling generation mechanism and a noise-aware generation method. To make generation closer to human writing patterns, this framework introduces a span-by-span generation flow that trains the model to predict semantically-complete spans consecutively rather than predicting word by word. Unlike existing pre-training methods, ERNIE-GEN incorporates multi-granularity target sampling to construct pre-training data, which enhances the correlation between encoder and decoder. Experimental results demonstrate that ERNIE-GEN achieves state-of-the-art results with a much smaller amount of pre-training data and parameters on a range of language generation tasks, including abstractive summarization (Gigaword and CNN/DailyMail), question generation (SQuAD), dialogue response generation (Persona-Chat) and generative question answering (CoQA). The source codes and pretrained models have been released at https://github. com/PaddlePaddle/ERNIE. * indicates equal contribution.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2001.11314")
get_code_for_paper("2001.11314")
have("2001.11314")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2001.11314.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2001.11314)
Connect an agent — have() is free.