We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
The pre-trained language models have achieved great successes in various natural language understanding (NLU) tasks due to its capacity to capture the deep contextualized information in text by pre-training on large-scale corpora. One of the fundamental components in pre-trained language models is the vocabulary, especially for training multilingual models on many different languages. In the technical report, we present our practices on training multilingual pre-trained language models with BBPE: Byte-Level BPE (i.e., Byte Pair Encoding). In the experiment, we adopted the architecture of NEZHA as the underlying pre-trained language model and the results show that NEZHA trained with byte-level subwords consistently outperforms Google multilingual BERT and vanilla NEZHA by a notable margin in several multilingual NLU tasks. We release the source code of our byte-level vocabulary building tools and the multilingual pre-trained language models.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2101.09469")
get_code_for_paper("2101.09469")
have("2101.09469")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2101.09469.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2101.09469)
Connect an agent — have() is free.