Li Zhang, Chris Callison-Burch, Yue Yang, Mark Yatskar, Artemis Panagopoulou, Qing Lyu
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Understanding what sequence of steps are needed to complete a goal can help artificial intelligence systems reason about human activities. Past work in NLP has examined the task of goal-step inference for text. We introduce the visual analogue. We propose the Visual Goal-Step Inference (VGSI) task, where a model is given a textual goal and must choose which of four images represents a plausible step towards that goal. With a new dataset harvested from wikiHow consisting of 772,277 images representing human actions, we show that our task is challenging for state-of-theart multimodal models. Moreover, the multimodal representation learned from our data can be effectively transferred to other datasets like HowTo100m, increasing the VGSI accuracy by 15 -20%. Our task will facilitate multimodal reasoning about procedural events.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2104.05845")
get_code_for_paper("2104.05845")
have("2104.05845")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2104.05845.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2104.05845)
Connect an agent — have() is free.