Chen Zhang, Haizhou Li, Luis Fernando D'haro, Qiquan Zhang, Thomas Friedrichs
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Recent model-based reference-free metrics for open-domain dialogue evaluation exhibit promising correlations with human judgment 1 . However, they either perform turn-level evaluation or look at a single dialogue quality dimension. One would expect a good evaluation metric to assess multiple quality dimensions at the dialogue level. To this end, we are motivated to propose a multi-dimensional dialogue-level metric, which consists of three sub-metrics with each targeting a specific dimension. The submetrics are trained with novel self-supervised objectives and exhibit strong correlations with human judgment for their respective dimensions. Moreover, we explore two approaches to combine the sub-metrics: metric ensemble and multitask learning. Both approaches yield a holistic metric that significantly outperforms individual sub-metrics. Compared to the existing state-of-the-art metric, the combined metrics achieve around 16% relative improvement on average across three high-quality dialoguelevel evaluation benchmarks.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2210.13832")
get_code_for_paper("2210.13832")
have("2210.13832")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2210.13832.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2210.13832)
Connect an agent — have() is free.