SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2409.13705 · EMNLP · 2024

Debiasing Text Safety Classifiers through a Fairness-Aware Ensemble Warning: This paper contains examples of potentially harmful text targeted towards identity groups

Aparna Joshi, Renee Shelby, Bhaktipriya Radharapu, Olivia Sturman, Piyush Kumar

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Increasing use of large language models (LLMs) demand performant guardrails to ensure the safety of inputs and outputs of LLMs. When these safeguards are trained on imbalanced data, they can learn the societal biases. We present a light-weight, post-processing method for mitigating counterfactual fairness in closed-source text safety classifiers. Our approach involves building an ensemble that not only outperforms the input classifiers and policy-aligns them, but also acts as a debiasing regularizer. We introduce two thresholdagnostic metrics to assess the counterfactual fairness of a model, and demonstrate how combining these metrics with Fair Data Reweighting (FDW) (Awasthi et al., 2020) helps mitigate biases. We create an expanded Open AI dataset (Markov et al., 2023), and a new templated LLM-generated dataset based on userprompts, both of which are counterfactually balanced across identity groups and cover four key areas of safety (Table 1); we will work towards publicly releasing these datasets 1 . Our results show that our approach improves counterfactual fairness with minimal impact on model performance.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2409.13705")
get_code_for_paper("2409.13705")
have("2409.13705")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2409.13705.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2409.13705.svg)](https://syntology.ai/paper/2409.13705)

Connect an agent — have() is free.