Lingfei Wu, Tengfei Ma, Shouling Ji, Hanlu Wu, Tariro Manyumwa
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Evaluation of a document summarization system has been a critical factor to impact the success of the summarization task. Previous approaches, such as ROUGE, mainly consider the informativeness of the assessed summary and require human-generated references for each test summary. In this work, we propose to evaluate the summary qualities without reference summaries by unsupervised contrastive learning. Specifically, we design a new metric which covers both linguistic qualities and semantic informativeness based on BERT. To learn the metric, for each summary, we construct different types of negative samples with respect to different aspects of the summary qualities, and train our model with a ranking loss. Experiments on Newsroom and CNN/Daily Mail demonstrate that our new evaluation method outperforms other metrics even without reference summaries. Furthermore, we show that our method is general and transferable across datasets.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2010.01781")
get_code_for_paper("2010.01781")
have("2010.01781")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2010.01781.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2010.01781)
Connect an agent — have() is free.