Ryan Lowe, Joelle Pineau, Koustuv Sinha, William Hamilton, Prasanna Parthasarathi, Jasmine Wang
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Evaluating the quality of a dialogue interaction between two agents is a difficult task, especially in open-domain chit-chat style dialogue. There have been recent efforts to develop automatic dialogue evaluation metrics, but most of them do not generalize to unseen datasets and/or need a human-generated reference response during inference, making it infeasible for online evaluation. Here, we propose an unreferenced automated evaluation metric that uses large pre-trained language models to extract latent representations of utterances, and leverages the temporal transitions that exist between them. We show that our model achieves higher correlation with human annotations in an online setting, while not requiring true responses for comparison during inference.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2005.00583")
get_code_for_paper("2005.00583")
have("2005.00583")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2005.00583.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2005.00583)
Connect an agent — have() is free.