SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2501.06892 · ACL · 2025

Language Fusion for Parameter-Efficient Cross-lingual Transfer

Marie-Francine Moens, Philipp Borchert, Jochen De Weerdt, Ivan Vulic

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Limited availability of multilingual text corpora for training language models often leads to poor performance on downstream tasks due to undertrained representation spaces for languages other than English. This 'under-representation' has motivated recent cross-lingual transfer methods to leverage the English representation space by e.g. mixing English and 'non-English' tokens at the input level or extending model parameters to accommodate new languages. However, these approaches often come at the cost of increased computational complexity. We propose Fusion for Language Representations (FLARE) in adapters, a novel method that enhances representation quality and downstream performance for languages other than English while maintaining parameter efficiency. FLARE integrates source and target language representations within low-rank (LoRA) adapters using lightweight linear transformations, maintaining parameter efficiency while improving transfer performance. A series of experiments across representative crosslingual natural language understanding tasks, including natural language inference, questionanswering and sentiment analysis, demonstrate FLARE's effectiveness. FLARE achieves performance improvements of 4.9% for Llama 3.1 and 2.2% for Gemma 2 compared to standard LoRA fine-tuning on question-answering tasks, as measured by the exact match metric. 1 method for cross-lingual language understanding.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2501.06892")
get_code_for_paper("2501.06892")
have("2501.06892")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2501.06892.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2501.06892.svg)](https://syntology.ai/paper/2501.06892)

Connect an agent — have() is free.