Yejin Choi, Lianhui Qin, Aditya Gupta, Shyam Upadhyay, Manaal Faruqui, Google Assistant, Luheng He
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Everyday conversations require understanding everyday events, which in turn, requires understanding temporal commonsense concepts interwoven with those events. Despite recent progress with massive pre-trained language models (LMs) such as T5 and GPT-3, their capability of temporal reasoning in dialogs remains largely under-explored. In this paper, we present the first study to investigate pre-trained LMs for their temporal reasoning capabilities in dialogs by introducing a new task and a crowd-sourced English challenge set, TIMEDIAL. We formulate TIME-DIAL as a multiple choice cloze task with over 1.1K carefully curated dialogs. Empirical results demonstrate that even the best performing models struggle on this task compared to humans, with 23 absolute points of gap in accuracy. Furthermore, our analysis reveals that the models fail to reason about dialog context correctly; instead, they rely on shallow cues based on existing temporal patterns in context, motivating future research for modeling temporal concepts in text and robust contextual reasoning about them. The dataset is publicly available at: https://github.com/ google-research-datasets/timedial.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2106.04571")
get_code_for_paper("2106.04571")
have("2106.04571")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2106.04571.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2106.04571)
Connect an agent — have() is free.