SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2302.14534 · EMNLP · 2023

Spacerini: Plug-and-play Search Engines with Pyserini and Hugging Face

Aleksandra Piktus, Christopher Akiki, Martin Potthast, Akintunde Oladipo, O. Ogundepo, Xinyu Crystina Zhang, Jimmy J. Lin

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 18 functions out of this paper's own repositories and ran 11 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
cookiecutter/cookiecutter canonical 9 of 13
castorini/hf-spacerini canonical 2 of 5
FunctionStatusWhere it lives
configure_logger Ran cookiecutter/cookiecutter/cookiecutter/log.py
code served (permissive licence) · get_code("c6962210cf21de5d")
create_tmp_repo_dir Ran cookiecutter/cookiecutter/cookiecutter/utils.py
code served (permissive licence) · get_code("d230c44fec10e142")
expand_abbreviations Ran cookiecutter/cookiecutter/cookiecutter/repository.py
code served (permissive licence) · get_code("7eaddd988c091a1a")
find_hook Ran cookiecutter/cookiecutter/cookiecutter/hooks.py
code served (permissive licence) · get_code("f1f202db52843cf5")
get_num_shards Ran castorini/hf-spacerini/src/spacerini/preprocess/utils.py
code served (permissive licence) · get_code("880ca31c02fd84ac")
is_copy_only_path Ran cookiecutter/cookiecutter/cookiecutter/generate.py
code served (permissive licence) · get_code("627d4d4a473ec3d5")
is_repo_url Ran cookiecutter/cookiecutter/cookiecutter/repository.py
code served (permissive licence) · get_code("541f278a32dba3e0")
is_zip_file Ran cookiecutter/cookiecutter/cookiecutter/repository.py
code served (permissive licence) · get_code("3fa141b4982471ef")
merge_configs Ran cookiecutter/cookiecutter/cookiecutter/config.py
code served (permissive licence) · get_code("7cd35c0d6160808f")
parse_args Ran castorini/hf-spacerini/src/spacerini/index/index.py
code served (permissive licence) · get_code("b49119e0346a7a76")
valid_hook Ran cookiecutter/cookiecutter/cookiecutter/hooks.py
code served (permissive licence) · get_code("409d801c82532737")
create_space_from_local Not yet run castorini/hf-spacerini/src/spacerini/frontend/space.py
code served (permissive licence) · get_code("aebe9a7adea92aba")
load_index_from_hub Not yet run castorini/hf-spacerini/src/spacerini/index/utils.py
code served (permissive licence) · get_code("475c48000dc4a34c")
push_index_to_hub Not yet run castorini/hf-spacerini/src/spacerini/index/utils.py
code served (permissive licence) · get_code("6b0c07ae42920502")
read_repo_password Not yet run cookiecutter/cookiecutter/cookiecutter/prompt.py
code served (permissive licence) · get_code("79ec09c0475e6eb0")
read_user_variable Not yet run cookiecutter/cookiecutter/cookiecutter/prompt.py
code served (permissive licence) · get_code("ddbdbda0aa3c7f27")
read_user_yes_no Not yet run cookiecutter/cookiecutter/cookiecutter/prompt.py
code served (permissive licence) · get_code("adac7f79ae4cfd6a")
simple_filter Not yet run cookiecutter/cookiecutter/cookiecutter/utils.py
code served (permissive licence) · get_code("8ea884ad3f0f2fb7")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We present Spacerini, a tool that integrates the Pyserini toolkit for reproducible information retrieval research with Hugging Face to enable the seamless construction and deployment of interactive search engines. Spacerini makes stateof-the-art sparse and dense retrieval models more accessible to non-IR practitioners while minimizing deployment effort. This is useful for NLP researchers who want to better understand and validate their research by performing qualitative analyses of training corpora, for IR researchers who want to demonstrate new retrieval models integrated into the growing Pyserini ecosystem, and for third parties reproducing the work of other researchers. Spacerini is open-source 1 and includes utilities for loading, preprocessing, indexing, and deploying search engines locally and remotely. We demonstrate a portfolio of 13 search engines created with Spacerini for different use cases. 2

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2302.14534")
get_code_for_paper("2302.14534")
have("2302.14534")

Connect an agent — have() is free.