Taylor Berg-Kirkpatrick, Eduard Hovy, Varun Gangal, Harsh Jhamtani
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Multiple different responses are often plausible for a given open domain dialog context. Prior work has shown the importance of having multiple valid reference responses for meaningful and robust automated evaluations. In such cases, common practice has been to collect more human written references. However, such collection can be expensive, time consuming, and not easily scalable. Instead, we propose a novel technique for automatically expanding a human generated reference to a set of candidate references. We fetch plausible references from knowledge sources, and adapt them so that they are more fluent in context of the dialog instance in question. More specifically, we use (1) a commonsense knowledge base to elicit a large number of plausible reactions given the dialog history (2) relevant instances retrieved from dialog corpus, using similar past as well as future contexts. We demonstrate that our automatically expanded reference sets lead to large improvements in correlations of automated metrics with human ratings of system outputs for DailyDialog dataset. 1
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2106.02833")
get_code_for_paper("2106.02833")
have("2106.02833")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2106.02833.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2106.02833)
Connect an agent — have() is free.