Yansong Feng, Dongyan Zhao, Shibo Hao, Quzhe Huang, Shengqi Zhu, Yuan Ye
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
DocRED is a widely used dataset for documentlevel relation extraction. In the large-scale annotation, a recommend-revise scheme is adopted to reduce the workload. Within this scheme, annotators are provided with candidate relation instances from distant supervision, and they then manually supplement and remove relational facts based on the recommendations. However, when comparing Do-cRED with a subset relabeled from scratch, we find that this scheme results in a considerable amount of false negative samples and an obvious bias towards popular entities and relations. Furthermore, we observe that the models trained on DocRED have low recall on our relabeled dataset and inherit the same bias in the training data. Through the analysis of annotators' behaviors, we figure out the underlying reason for the problems above: the scheme actually discourages annotators from supplementing adequate instances in the revision phase. We appeal to future research to take into consideration the issues with the recommend-revise scheme when designing new models and annotation schemes. The relabeled dataset is released at https://github. com/AndrewZhe/Revisit-DocRED, to serve as a more reliable test set of document RE models.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2204.07980")
get_code_for_paper("2204.07980")
have("2204.07980")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2204.07980.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2204.07980)
Connect an agent — have() is free.