Bang An, Benyou Wang, Haizhou Li, Juhao Liang, Zhenyang Cai, Jianqing Zhu, Huang Huang, Kewei Zong, Abdulmohsen Alharthi, Juncai He, Lian Zhang, Jinchao Xu
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
The alignment of large language models (LLMs) is critical for developing effective and safe language models. Traditional approaches focus on aligning models during the instruction tuning or reinforcement learning stages, referred to in this paper as 'post alignment'. We argue that alignment during the pre-training phase, which we term 'native alignment', warrants investigation. Native alignment aims to prevent unaligned content from the beginning, rather than relying on posthoc processing. This approach leverages extensively aligned pre-training data to enhance the effectiveness and usability of pre-trained models. Our study specifically explores the application of native alignment in the context of Arabic LLMs. We conduct comprehensive experiments and ablation studies to evaluate the impact of native alignment on model performance and alignment stability. Additionally, we release open-source Arabic LLMs that demonstrate state-of-the-art performance on various benchmarks, providing significant benefits to the Arabic LLM community. 1
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2412.03253")
get_code_for_paper("2412.03253")
have("2412.03253")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2412.03253.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2412.03253)
Connect an agent — have() is free.