SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2204.01334 · ACL Findings · 2022

Efficient, Uncertainty-based Moderation of Neural Networks Text Classifiers

Jakob Andersen, Walid Maalej

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

To maximize the accuracy and increase the overall acceptance of text classifiers, we propose a framework for the efficient, inoperation moderation of classifiers' output. Our framework focuses on use cases in which F1-scores of modern Neural Networks classifiers (ca. 90%) are still inapplicable in practice. We suggest a semi-automated approach that uses prediction uncertainties to pass unconfident, probably incorrect classifications to human moderators. To minimize the workload, we limit the human moderated data to the point where the accuracy gains saturate and further human effort does not lead to substantial improvements. A series of benchmarking experiments based on three different datasets and three state-of-the-art classifiers show that our framework can improve the classification F1scores by 5.1 to 11.2% (up to approx. 98 to 99%), while reducing the moderation load up to 73.3% compared to a random moderation.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2204.01334")
get_code_for_paper("2204.01334")
have("2204.01334")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2204.01334.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2204.01334.svg)](https://syntology.ai/paper/2204.01334)

Connect an agent — have() is free.