SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2303.08128 · ICCV · 2023

ViperGPT: Visual Inference via Python Execution for Reasoning

Carl Vondrick, Dídac Surís, Sachit Menon

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Answering visual queries is a complex task that requires both visual processing and reasoning. End-to-end models, the dominant approach for this task, do not explicitly differentiate between the two, limiting interpretability and generalization. Learning modular programs presents a promising alternative, but has proven challenging due to the difficulty of learning both the programs and modules simultaneously. We introduce ViperGPT, a framework that leverages codegeneration models to compose vision-and-language models into subroutines to produce a result for any query. ViperGPT utilizes a provided API to access the available modules, and composes them by generating Python code that is later executed. This simple approach requires no further training, and achieves state-of-the-art results across various complex visual tasks. * Equal contribution. Order determined via coin flip and may be listed either way.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2303.08128")
get_code_for_paper("2303.08128")
have("2303.08128")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2303.08128.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2303.08128.svg)](https://syntology.ai/paper/2303.08128)

Connect an agent — have() is free.