Théo Gigant, Camille Guinaudeau, Marc Decombas, Frederic Dufaux
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. If it links a repository, it is listed below.
Automatic metrics are used as proxies to evaluate abstractive summarization systems when human annotations are too expensive. To be useful, these metrics should be fine-grained, show a high correlation with human annotations, and ideally be independent of reference quality; however, most standard evaluation metrics for summarization are reference-based, and existing reference-free metrics correlate poorly with relevance, especially on summaries of longer documents. In this paper, we introduce a reference-free metric that correlates well with human evaluated relevance, while being very cheap to compute. We show that this metric can also be used alongside reference-based metrics to improve their robustness in low quality reference settings.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2410.10867")
get_code_for_paper("2410.10867")
have("2410.10867")
Connect an agent — have() is free.