SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2306.08952 · ACL · 2023

Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Models

Lidong Bing, Hwee Ng, Qingyu Tan

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Reasoning about time is of fundamental importance. Many facts are time-dependent. For example, athletes change teams from time to time, and different government officials are elected periodically. Previous time-dependent question answering (QA) datasets tend to be biased in either their coverage of time spans or question types. In this paper, we introduce a comprehensive probing dataset TEMPREASON to evaluate the temporal reasoning capability of large language models. Our dataset includes questions of three temporal reasoning levels. In addition, we also propose a novel learning framework to improve the temporal reasoning capability of large language models, based on temporal span extraction and time-sensitive reinforcement learning. We conducted experiments in closed book QA, open book QA, and reasoning QA settings and demonstrated the effectiveness of our approach 1 . * Qingyu Tan is under the Joint PhD Program between Alibaba and NUS. † Corresponding author. 1 Our code and data are released on https://github.com/ DAMO-NLP-SG/TempReason

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2306.08952")
get_code_for_paper("2306.08952")
have("2306.08952")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2306.08952.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2306.08952.svg)](https://syntology.ai/paper/2306.08952)

Connect an agent — have() is free.