Alexander Fraser, Jindřich Libovický, Katharina Hämmerl
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Static and contextual multilingual embeddings have complementary strengths. Static embeddings, while less expressive than contextual language models, can be more straightforwardly aligned across multiple languages. We combine the strengths of static and contextual models to improve multilingual representations. We extract static embeddings for 40 languages from XLM-R, validate those embeddings with cross-lingual word retrieval, and then align them using VecMap. This results in high-quality, highly multilingual static embeddings. Then we apply a novel continued pre-training approach to XLM-R, leveraging the high quality alignment of our static embeddings to better align the representation space of XLM-R. We show positive results for multiple complex semantic tasks. We release the static embeddings and the continued pretraining code. 1 Unlike most previous work, our continued pre-training approach does not require parallel text.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2203.09326")
get_code_for_paper("2203.09326")
have("2203.09326")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2203.09326.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2203.09326)
Connect an agent — have() is free.