SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2109.02846 · EMNLP · 2021

Datasets: A Community Library for Natural Language Processing

Teven Le Scao, Canwen Xu, Quentin Lhoest, Yacine Jernite, Thomas Wolf, Victor Sanh, Gunjan Chhablani, Lewis Tunstall, Julien Chaumond, Stas Bekman, Suraj Patil, Joe Davison, and 20 more

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

The scale, variety, and quantity of publiclyavailable NLP datasets has grown rapidly as researchers propose new tasks, larger models, and novel benchmarks. Datasets is a community library for contemporary NLP designed to support this ecosystem. Datasets aims to standardize end-user interfaces, versioning, and documentation, while providing a lightweight front-end that behaves similarly for small datasets as for internet-scale corpora. The design of the library incorporates a distributed, community-driven approach to adding datasets and documenting usage. After a year of development, the library now includes more than 650 unique datasets, has more than 250 contributors, and has helped support a variety of novel crossdataset research projects and shared tasks. The library is available at https://github. com/huggingface/datasets.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2109.02846")
get_code_for_paper("2109.02846")
have("2109.02846")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2109.02846.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2109.02846.svg)](https://syntology.ai/paper/2109.02846)

Connect an agent — have() is free.