SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2401.10353 · EACL Findings · 2024

Inconsistent dialogue responses and how to recover from them

Dong Yu, Linfeng Song, Jin Lifeng, Haitao Mi, Mian Zhang

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 6 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
mianzhang/cider canonical 6 of 6
FunctionStatusWhere it lives
combine_one_seq Ran mianzhang/cider/src/utils.py
pointer only (licence: NONE) · get_code("7b91bbbc80876a25")
flat_array Ran mianzhang/cider/src/utils.py
pointer only (licence: NONE) · get_code("1863a6f51080495b")
read_cdconv Ran mianzhang/cider/src/data_utils.py
pointer only (licence: NONE) · get_code("2728760bdc96e355")
read_single_file Ran mianzhang/cider/src/data_utils.py
pointer only (licence: NONE) · get_code("0bc958a2294e3162")
read_stance Ran mianzhang/cider/src/data_utils.py
pointer only (licence: NONE) · get_code("32a50cf8331ae986")
tokenize Ran mianzhang/cider/src/utils.py
pointer only (licence: NONE) · get_code("ac28a2d89a01c888")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

One critical issue for chat systems is to stay consistent about preferences, opinions, beliefs and facts of itself, which has been shown a difficult problem. In this work, we study methods to assess and bolster utterance consistency of chat systems. A dataset is first developed for studying the inconsistencies, where inconsistent dialogue responses, explanations of the inconsistencies, and recovery utterances are authored by annotators. This covers the life span of inconsistencies, namely introduction, understanding, and resolution. Building on this, we introduce a set of tasks centered on dialogue consistency, specifically focused on its detection and resolution. Our experimental findings indicate that our dataset significantly helps the progress in identifying and resolving conversational inconsistencies, and current popular large language models like ChatGPT which are good at resolving inconsistencies however still struggle with detection. 1

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2401.10353")
get_code_for_paper("2401.10353")
have("2401.10353")

Connect an agent — have() is free.