SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2004.14356 · EMNLP · 2020

AXCELL: Automatic Extraction of Results from Machine Learning Papers

Sebastian Ruder, Sebastian Riedel, Pontus Stenetorp, Marcin Kardas, Robert Stojnic, Ross Taylor, Piotr Czapla

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 14 functions out of this paper's own repositories and ran 12 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
paperswithcode/axcell canonical 12 of 14
FunctionStatusWhere it lives
annotations Ran paperswithcode/axcell/axcell/helpers/optimize.py
code served (permissive licence) · get_code("53bc9344cc348810")
estimate_noises Ran paperswithcode/axcell/axcell/helpers/optimize.py
code served (permissive licence) · get_code("3387a9ba01a1d9a2")
load_references Ran paperswithcode/axcell/axcell/helpers/cache.py
code served (permissive licence) · get_code("83f38dae7103f51b")
load_structure Ran paperswithcode/axcell/axcell/helpers/cache.py
code served (permissive licence) · get_code("ad584250d25d1fb1")
load_tags Ran paperswithcode/axcell/axcell/helpers/cache.py
code served (permissive licence) · get_code("e150c098756c6b8e")
norm_score_str Ran paperswithcode/axcell/axcell/helpers/evaluate.py
code served (permissive licence) · get_code("b75165430b90d460")
precision Ran paperswithcode/axcell/axcell/helpers/evaluate.py
code served (permissive licence) · get_code("7fc6d3a39db0d9de")
read_arxiv_papers Ran paperswithcode/axcell/axcell/helpers/datasets.py
code served (permissive licence) · get_code("4360ff7024678437")
read_tables_annotations Ran paperswithcode/axcell/axcell/helpers/datasets.py
code served (permissive licence) · get_code("02b6a005a3d5a4bb")
recall Ran paperswithcode/axcell/axcell/helpers/evaluate.py
code served (permissive licence) · get_code("f02f73aa554eecb3")
ro_bind Ran paperswithcode/axcell/axcell/helpers/latex_converter.py
code served (permissive licence) · get_code("f8b1fe756d77cd7e")
rw_bind Ran paperswithcode/axcell/axcell/helpers/latex_converter.py
code served (permissive licence) · get_code("f05aaf2be87524df")
estimate_context_noise Not yet run paperswithcode/axcell/axcell/helpers/optimize.py
code served (permissive licence) · get_code("86fb74ea9103b92b")
table_to_html Not yet run paperswithcode/axcell/axcell/helpers/jupyter.py
code served (permissive licence) · get_code("4767e70065da93cd")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Tracking progress in machine learning has become increasingly difficult with the recent explosion in the number of papers. In this paper, we present AXCELL, an automatic machine learning pipeline for extracting results from papers. AXCELL uses several novel components, including a table segmentation subtask, to learn relevant structural knowledge that aids extraction. When compared with existing methods, our approach significantly improves the state of the art for results extraction. We also release a structured, annotated dataset for training models for results extraction, and a dataset for evaluating the performance of models on this task. Lastly, we show the viability of our approach enables it to be used for semi-automated results extraction in production, suggesting our improvements make this task practically viable for the first time. Code is available on GitHub. 1 Back-translation . . .

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2004.14356")
get_code_for_paper("2004.14356")
have("2004.14356")

Connect an agent — have() is free.