We lifted 2 functions out of this paper's own repositories and ran 0 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| google-research/metricx | canonical | 0 of 2 |
| Function | Status | Where it lives |
|---|---|---|
| get_dataset | Not yet run | google-research/metricx/metricx23/predict.py code served (permissive licence) · get_code("96776db63f1e0113") |
| get_dataset | Not yet run | google-research/metricx/metricx24/predict.py code served (permissive licence) · get_code("0298705924fae843") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
In this paper, we present the MetricX-24 submissions to the WMT24 Metrics Shared Task and provide details on the improvements we made over the previous version of MetricX. Our primary submission is a hybrid reference-based/-free metric, which can score a translation irrespective of whether it is given the source segment, the reference, or both. The metric is trained on previous WMT data in a two-stage fashion, first on the DA ratings only, then on a mixture of MQM and DA ratings. The training set in both stages is augmented with synthetic examples that we created to make the metric more robust to several common failure modes, such as fluent but unrelated translation, or undertranslation. We demonstrate the benefits of the individual modifications via an ablation study, and show a significant performance increase over MetricX-23 on the WMT23 MQM ratings, as well as our new synthetic challenge set.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2410.03983")
get_code_for_paper("2410.03983")
have("2410.03983")
Connect an agent — have() is free.