SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2101.04727 · 2021

Latent Alignment of Procedural Concepts in Multimodal Recipes

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We propose a novel alignment mechanism to deal with procedural reasoning on a newly released multimodal QA dataset, named RecipeQA. Our model is solving the textual cloze task which is a reading comprehension on a recipe containing images and instructions. We exploit the power of attention networks, cross-modal representations, and a latent alignment space between instructions and candidate answers to solve the problem. We introduce constrained max-pooling which refines the max-pooling operation on the alignment matrix to impose disjoint constraints among the outputs of the model. Our evaluation result indicates a 19\% improvement over the baselines.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2101.04727")
get_code_for_paper("2101.04727")
have("2101.04727")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2101.04727.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2101.04727.svg)](https://syntology.ai/paper/2101.04727)

Connect an agent — have() is free.