SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2202.06991 · NeurIPS · 2022

Transformer Memory as a Differentiable Search Index

William Cohen, Yi Tay, Dara Bahri, Donald Metzler, Zhe Zhao, Zhen Qin, Mostafa Dehghani, Tal Schuster, Vinh Tran, Jai Gupta, Jianmo Ni, Harsh Mehta, and 1 more

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

In this paper, we demonstrate that information retrieval can be accomplished with a single Transformer, in which all information about the corpus is encoded in the parameters of the model. To this end, we introduce the Differentiable Search Index (DSI), a new paradigm that learns a text-to-text model that maps string queries directly to relevant docids; in other words, a DSI model answers queries directly using only its parameters, dramatically simplifying the whole retrieval process. We study variations in how documents and their identifiers are represented, variations in training procedures, and the interplay between models and corpus sizes. Experiments demonstrate that given appropriate design choices, DSI significantly outperforms strong baselines such as dual encoder models. Moreover, DSI demonstrates strong generalization capabilities, outperforming a BM25 baseline in a zero-shot setup.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2202.06991")
get_code_for_paper("2202.06991")
have("2202.06991")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2202.06991.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2202.06991.svg)](https://syntology.ai/paper/2202.06991)

Connect an agent — have() is free.