We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Natural language generation (NLG) systems are commonly evaluated using n-gram overlap measures (e.g. BLEU, ROUGE). These measures do not directly capture semantics or speaker intentions, and so they often turn out to be misaligned with our true goals for NLG. In this work, we argue instead for communication-based evaluations: assuming the purpose of an NLG system is to convey information to a reader/listener, we can directly evaluate its effectiveness at this task using the Rational Speech Acts model of pragmatic language use. We illustrate with a color reference dataset that contains descriptions in pre-defined quality categories, showing that our method better aligns with these quality categories than do any of the prominent n-gram overlap methods.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("1909.07290")
get_code_for_paper("1909.07290")
have("1909.07290")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/1909.07290.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/1909.07290)
Connect an agent — have() is free.