Ziyi Yang, Eric Darve, Yinfei Yang, Daniel Cer
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Language agnostic and semantic-language information isolation is an emerging research direction for multilingual representations models. We explore this problem from a novel angle of geometric algebra and semantic space. A simple but highly effective method "Language Information Removal (LIR)" factors out language identity information from semantic related components in multilingual representations pre-trained on multi-monolingual data. A post-training and model-agnostic method, LIR only uses simple linear operations, e.g. matrix factorization and orthogonal projection. LIR reveals that for weak-alignment multilingual systems, the principal components of semantic spaces primarily encodes language identity information. We first evaluate the LIR on a cross-lingual question answer retrieval task (LAReQA), which requires the strong alignment for the multilingual embedding space. Experiment shows that LIR is highly effectively on this task, yielding almost 100% relative improvement in MAP for weakalignment models. We then evaluate the LIR on Amazon Reviews and XEVAL dataset, with the observation that removing language information is able to improve the cross-lingual transfer performance.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2109.04727")
get_code_for_paper("2109.04727")
have("2109.04727")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2109.04727.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2109.04727)
Connect an agent — have() is free.