SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2303.14770 · EMNLP · 2023

Koala: An Index for Quantifying Overlaps with Pre-training Corpora

Xuanli He, Gholamreza Haffari, Thuy-Trang Vu, Ehsan Shareghi

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 3 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
ekzhu/datasketch canonical 3 of 3
FunctionStatusWhere it lives
optimal_partitions Ran ekzhu/datasketch/datasketch/lshensemble_partition.py
code served (permissive licence) · get_code("bd7d7a9c3a6709ee")
sha1_hash32 Ran ekzhu/datasketch/datasketch/hashfunc.py
code served (permissive licence) · get_code("6dc9da353c1b85b0")
sha1_hash64 Ran ekzhu/datasketch/datasketch/hashfunc.py
code served (permissive licence) · get_code("9c25f231de344715")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

In very recent years more attention has been placed on probing the role of pre-training data in Large Language Models (LLMs) downstream behaviour. Despite the importance, there is no public tool that supports such analysis of pre-training corpora at large scale. To help research in this space, we launch Koala, a searchable index over large pretraining corpora using lossless compressed suffix arrays with highly efficient compression rate and search support. In its first release we index the public proportion of OPT 175B, GPT-3, GPT-Neo, GPT-Neo, LLaMA, BERT, ELECTRA, RoBERTA, XLNet pre-training corpora. Koala provides a framework to do forensic analysis on the current and future benchmarks as well as to assess the degree of memorization in the output from the LLMs. Koala is available for public use at https: //koala-index.erc.monash.edu/.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2303.14770")
get_code_for_paper("2303.14770")
have("2303.14770")

Connect an agent — have() is free.