Benjamin Hsu, Anna Currey, Xing Niu, Maria Nȃdejde, Georgiana Dinu, Raghavendra Pappagari, Mia Mayer, Stanislas Lauly
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
As generic machine translation (MT) quality has improved, the need for targeted benchmarks that explore fine-grained aspects of quality has increased (Freitag et al., 2021;Isabelle et al., 2017). In particular, gender accuracy in translation (Choubey et al., 2021; Saunders and Byrne, 2020) can have implications in terms of output fluency, translation accuracy, and ethics. In this paper, we introduce MT-GenEval, a benchmark for evaluating gender accuracy in translation from English into eight widely-spoken languages. MT-GenEval complements existing benchmarks by providing realistic, gender-balanced, counterfactual data in eight language pairs where the gender of individuals is unambiguous in the input segment, including multi-sentence segments requiring inter-sentential gender agreement. Our data and code is publicly available under a CC BY SA 3.0 license. 1
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2211.01355")
get_code_for_paper("2211.01355")
have("2211.01355")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2211.01355.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2211.01355)
Connect an agent — have() is free.