SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2009.07690 · EMNLP · 2020

Reasoning about Goals, Steps, and Temporal Ordering with WikiHow

Li Zhang, Chris Callison-Burch, Qing Lyu

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We propose a suite of reasoning tasks on two types of relations between procedural events: GOAL-STEP relations ("learn poses" is a step in the larger goal of "doing yoga") and STEP-STEP TEMPORAL relations ("buy a yoga mat" typically precedes "learn poses"). We introduce a dataset targeting these two relations based on wikiHow, a website of instructional how-to articles. Our human-validated test set serves as a reliable benchmark for commonsense inference, with a gap of about 10% to 20% between the performance of state-ofthe-art transformer models and human performance. Our automatically-generated training set allows models to effectively transfer to outof-domain tasks requiring knowledge of procedural events, with greatly improved performances on SWAG, Snips, and Story Cloze Test in zero-and few-shot settings.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2009.07690")
get_code_for_paper("2009.07690")
have("2009.07690")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2009.07690.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2009.07690.svg)](https://syntology.ai/paper/2009.07690)

Connect an agent — have() is free.