SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2103.11332 · NAACL · 2021

AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization

Pascale Fung, Zihan Liu, Tiezheng Yu

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

State-of-the-art abstractive summarization models generally rely on extensive labeled data, which lowers their generalization ability on domains where such data are not available. In this paper, we present a study of domain adaptation for the abstractive summarization task across six diverse target domains in a low-resource setting. Specifically, we investigate the second phase of pre-training on large-scale generative models under three different settings: 1) source domain pre-training; 2) domain-adaptive pre-training; and 3) taskadaptive pre-training. Experiments show that the effectiveness of pre-training is correlated with the similarity between the pre-training data and the target domain task. Moreover, we find that continuing pre-training could lead to the pre-trained model's catastrophic forgetting, and a learning method with less forgetting can alleviate this issue. Furthermore, results illustrate that a huge gap still exists between the low-resource and high-resource settings, which highlights the need for more advanced domain adaptation methods for the abstractive summarization task. 1 * * Equal contributions. Listing order is random. 1 The code and data are released at: https://github. com/TysonYu/AdaptSum

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2103.11332")
get_code_for_paper("2103.11332")
have("2103.11332")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2103.11332.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2103.11332.svg)](https://syntology.ai/paper/2103.11332)

Connect an agent — have() is free.