SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2306.12907 · ACL · 2023

xSIM++: An Improved Proxy to Bitext Mining Performance for Low-Resource Languages

Meta Research, Mingda Chen, Onur Çelebi, Kevin Heffernan, Holger Schwenk, Alex Mourachko

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We introduce a new proxy score for evaluating bitext mining based on similarity in a multilingual embedding space: xsim++. In comparison to xsim, this improved proxy leverages rulebased approaches to extend English sentences in any evaluation set with synthetic, hard-todistinguish examples which more closely mirror the scenarios we encounter during largescale mining. We validate this proxy by running a significant number of bitext mining experiments for a set of low-resource languages, and subsequently train NMT systems on the mined data. In comparison to xsim, we show that xsim++ is better correlated with the downstream BLEU scores of translation systems trained on mined bitexts, providing a reliable proxy of bitext mining performance without needing to run expensive bitext mining pipelines. xsim++ also reports performance for different error types, offering more fine-grained feedback for model development.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2306.12907")
get_code_for_paper("2306.12907")
have("2306.12907")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2306.12907.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2306.12907.svg)](https://syntology.ai/paper/2306.12907)

Connect an agent — have() is free.