Dietrich Klakow, Marius Mosbach, Mor Geva, Jesujoba Alabi, Matan Eyal
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We analyze the operation of transformer language adapters, which are small modules trained on top of a frozen language model to adapt its predictions to new target languages. We show that adapted predictions mostly evolve in the source language the model was trained on, while the target language becomes pronounced only in the very last layers of the model. Moreover, the adaptation process is gradual and distributed across layers, where it is possible to skip small groups of adapters without decreasing adaptation performance. Last, we show that adapters operate on top of the model's frozen representation space while largely preserving its structure, rather than on an "isolated" subspace. Our findings provide a deeper view into the adaptation process of language models to new languages, showcasing the constraints imposed on it by the underlying model and introduces practical implications to enhance its efficiency. 1
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2402.13137")
get_code_for_paper("2402.13137")
have("2402.13137")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2402.13137.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2402.13137)
Connect an agent — have() is free.