Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, Canwen Xu
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
In this paper, we propose Sequence Span Rewriting (SSR), a self-supervised task for sequence-to-sequence (Seq2Seq) pre-training. SSR learns to refine the machine-generated imperfect text spans into ground truth text. SSR provides more fine-grained and informative supervision in addition to the original textinfilling objective. Compared to the prevalent text infilling objectives for Seq2Seq pretraining, SSR is naturally more consistent with many downstream generation tasks that require sentence rewriting (e.g., text summarization, question generation, grammatical error correction, and paraphrase generation). We conduct extensive experiments by using SSR to improve the typical Seq2Seq pre-trained model T5 in a continual pre-training setting and show substantial improvements over T5 on various natural language generation tasks. 1
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2101.00416")
get_code_for_paper("2101.00416")
have("2101.00416")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2101.00416.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2101.00416)
Connect an agent — have() is free.