Haoran Sun, Ling Yang, Lei Lei, Xiong, Renren Jin, Leiyu Pan, Shaoyang Xu, Menglong Supryadi, Jiangcun Cui, Yikun Du, Juesi Shi, Shaolin Xiao, and 1 more
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Large language models (LLMs) have demonstrated prowess in a wide range of tasks. However, many LLMs exhibit significant performance discrepancies between high-and lowresource languages. To mitigate this challenge, we present FuxiTranyu, an open-source multilingual LLM, which is designed to satisfy the need of the research community for balanced and high-performing multilingual capabilities. The base model, FuxiTranyu-8B, features 8 billion parameters and is trained from scratch on meticulously balanced multilingual data that contains 600 billion tokens covering 43 natural languages and 16 programming languages. We also develop two instructiontuned models: FuxiTranyu-8B-SFT which is fine-tuned on a diverse multilingual instruction dataset, and FuxiTranyu-8B-DPO which is further refined with DPO on a preference dataset for enhanced alignment ability. Extensive experiments on a wide range of multilingual benchmarks demonstrate the competitive performance of FuxiTranyu against existing multilingual LLMs, e.g., BLOOM-7B, PolyLM-13B, and Mistral-7B-Instruct. Both neuron and representation interpretability analyses reveal that FuxiTranyu achieves consistent multilingual representations across languages. To promote further research into multilingual LLMs, we release both the base and instruction-tuned Fux-iTranyu models together with 58 pre-training checkpoints at HuggingFace 1 and Github. 2 * Corresponding author. 1
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2408.06273")
get_code_for_paper("2408.06273")
have("2408.06273")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2408.06273.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2408.06273)
Connect an agent — have() is free.