Christof Monz, Bill Byrne, David Stap, Amazon Agi, Eva Hasler, Ke Tran
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Fine-tuning large language models (LLMs) for machine translation has shown improvements in overall translation quality. However, it is unclear what is the impact of fine-tuning on desirable LLM behaviors that are not present in neural machine translation models, such as steerability, inherent document-level translation abilities, and the ability to produce less literal translations. We perform an extensive translation evaluation on the LLaMA and Falcon family of models with model size ranging from 7 billion up to 65 billion parameters. Our results show that while fine-tuning improves the general translation quality of LLMs, several abilities degrade. In particular, we observe a decline in the ability to perform formality steering, to produce technical translations through few-shot examples, and to perform documentlevel translation. On the other hand, we observe that the model produces less literal translations after fine-tuning on parallel data. We show that by including monolingual data as part of the fine-tuning data we can maintain the abilities while simultaneously enhancing overall translation quality. Our findings emphasize the need for fine-tuning strategies that preserve the benefits of LLMs for machine translation.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2405.20089")
get_code_for_paper("2405.20089")
have("2405.20089")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2405.20089.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2405.20089)
Connect an agent — have() is free.