SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2209.07442 · ACL · 2022

Automatic Error Analysis for Document-level Information Extraction

Jiayuan Gu, Kejian Shi, Xinya Du, Claire Cardie, Barry Wang, Aliva Das, Thomas Porter

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

tasks have recently begun to be revisited in earnest using the end-to-end neural network techniques that have been successful on their sentence-level IE counterparts. Evaluation of the approaches, however, has been limited in a number of dimensions. In particular, the precision/recall/F1 scores typically reported provide few insights on the range of errors the models make. We build on the work of Kummerfeld and Klein (2013) to propose a transformation-based framework for automating error analysis in document-level event and (N-ary) relation extraction. We employ our framework to compare two state-of-theart document-level template-filling approaches on datasets from three domains; and then, to gauge progress in IE since its inception 30 years ago, vs. four systems from the MUC-4 (1992) evaluation. 1 * These authors contributed equally to this work. 1 Our code for the error analysis tool and its output on different model predictions are available at https://github. com/IceJinx33/auto-err-template-fill/.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2209.07442")
get_code_for_paper("2209.07442")
have("2209.07442")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2209.07442.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2209.07442.svg)](https://syntology.ai/paper/2209.07442)

Connect an agent — have() is free.