SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 1809.00786 · 2018

Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We propose to decompose instruction execution to goal prediction and action generation. We design a model that maps raw visual observations to goals using LINGUNET, a language-conditioned image generation network, and then generates the actions required to complete them. Our model is trained from demonstration only without external resources. To evaluate our approach, we introduce two benchmarks for instruction following: LANI, a navigation task; and CHAI, where an agent executes household instructions. Our evaluation demonstrates the advantages of our model decomposition, and illustrates the challenges posed by our new benchmarks.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("1809.00786")
get_code_for_paper("1809.00786")
have("1809.00786")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/1809.00786.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/1809.00786.svg)](https://syntology.ai/paper/1809.00786)

Connect an agent — have() is free.