Rui Yan, Juntao Li, Lidong Bing, Hwee Ng, Ruidan He, Hai Ye
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Recent research indicates that pretraining crosslingual language models on large-scale unlabeled texts yields significant performance improvements over various cross-lingual and low-resource tasks. Through training on one hundred languages and terabytes of texts, cross-lingual language models have proven to be effective in leveraging high-resource languages to enhance low-resource language processing and outperform monolingual models. In this paper, we further investigate the cross-lingual and cross-domain (CLCD) setting when a pretrained cross-lingual language model needs to adapt to new domains. Specifically, we propose a novel unsupervised feature decomposition method that can automatically extract domain-specific features and domain-invariant features from the entangled pretrained cross-lingual representations, given unlabeled raw texts in the source language. Our proposed model leverages mutual information estimation to decompose the representations computed by a cross-lingual model into domain-invariant and domain-specific parts. Experimental results show that our proposed method achieves significant performance improvements over the state-of-the-art pretrained crosslingual language model in the CLCD setting.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2011.11499")
get_code_for_paper("2011.11499")
have("2011.11499")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2011.11499.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2011.11499)
Connect an agent — have() is free.