SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2011.03992 · 2020

A Gold Standard Methodology for Evaluating Accuracy in Data-To-Text Systems

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Most Natural Language Generation systems need to produce accurate texts. We propose a methodology for high-quality human evaluation of the accuracy of generated texts, which is intended to serve as a gold-standard for accuracy evaluations of data-to-text systems. We use our methodology to evaluate the accuracy of computer generated basketball summaries. We then show how our gold standard evaluation can be used to validate automated metrics

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2011.03992")
get_code_for_paper("2011.03992")
have("2011.03992")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2011.03992.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2011.03992.svg)](https://syntology.ai/paper/2011.03992)

Connect an agent — have() is free.