We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Counterfactual text generation aims to minimally change a text, such that it is classified differently. Judging advancements in method development for counterfactual text generation is hindered by a non-uniform usage of data sets and metrics in related work. We propose CEval, a benchmark for comparing counterfactual text generation methods. CEval unifies counterfactual and text quality metrics, includes common counterfactual datasets with human annotations, standard baselines (MICE, GDBA, CREST) and the open-source language model LLAMA-2. Our experiments found no perfect method for generating counterfactual text. Methods that excel at counterfactual metrics often produce lower-quality text while LLMs with simple prompts generate high-quality text but struggle with counterfactual criteria. By making CEval available as an open-source Python library, we encourage the community to contribute more methods and maintain consistent evaluation in future work.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2404.17475")
get_code_for_paper("2404.17475")
have("2404.17475")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2404.17475.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2404.17475)
Connect an agent — have() is free.