SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2104.05232 · NAACL · 2021

Double Perturbation: On the Robustness of Robustness and Counterfactual Bias Evaluation

Huan Zhang, Cho-Jui Hsieh, Kai-Wei Chang, Chong Zhang, Jieyu Zhao

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Robustness and counterfactual bias are usually evaluated on a test dataset. However, are these evaluations robust? If the test dataset is perturbed slightly, will the evaluation results keep the same? In this paper, we propose a "double perturbation" framework to uncover model weaknesses beyond the test dataset. The framework first perturbs the test dataset to construct abundant natural sentences similar to the test data, and then diagnoses the prediction change regarding a single-word substitution. We apply this framework to study two perturbation-based approaches that are used to analyze models' robustness and counterfactual bias in English. (1) For robustness, we focus on synonym substitutions and identify vulnerable examples where prediction can be altered. Our proposed attack attains high success rates (96.0%-99.8%) in finding vulnerable examples on both original and robustly trained CNNs and Transformers. (2) For counterfactual bias, we focus on substituting demographic tokens (e.g., gender, race) and measure the shift of the expected prediction among constructed sentences. Our method is able to reveal the hidden model biases not directly shown in the test dataset. Our code is available at https://github.com/chong-z/ nlp-second-order-attack.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2104.05232")
get_code_for_paper("2104.05232")
have("2104.05232")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2104.05232.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2104.05232.svg)](https://syntology.ai/paper/2104.05232)

Connect an agent — have() is free.