Qian Liu, Min Lin, Jia Guo, Guangtao Zeng, Longxu Dou, Jiahui Zhou, Wei Lu, Ziqi Jin, Xin Mao
We lifted 12 functions out of this paper's own repositories and ran 8 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| epfllm/megatron-llm | canonical | 7 of 11 |
| sail-sg/sailor-llm | canonical | 1 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| all_gather_item | Ran | epfllm/megatron-llm/megatron/dist_signal_handler.py pointer only (licence: NOASSERTION) · get_code("684c66f5cec0794e") |
| allocate_mem_buff | Ran | epfllm/megatron-llm/megatron/memory.py pointer only (licence: NONE) · get_code("db8deebff0d5256c") |
| assert_viewless_tensor | Ran | epfllm/megatron-llm/megatron/core/utils.py pointer only (licence: NOASSERTION) · get_code("cc64b6ee963afc52") |
| build_num_microbatches_calculator | Ran | epfllm/megatron-llm/megatron/microbatches.py pointer only (licence: NOASSERTION) · get_code("99baeb4271f72548") |
| divide | Ran | epfllm/megatron-llm/megatron/core/utils.py pointer only (licence: NONE) · get_code("33de97c702d46371") |
| first_option_parse | Ran | sail-sg/sailor-llm/eval/sailor_text_postprocessors.py code served (permissive licence) · get_code("84fc895d549d7b93") |
| get_device | Ran | epfllm/megatron-llm/megatron/dist_signal_handler.py pointer only (licence: NOASSERTION) · get_code("11ee5870494a1570") |
| make_viewless_tensor | Ran | epfllm/megatron-llm/megatron/core/utils.py pointer only (licence: NOASSERTION) · get_code("9114f74bf4588d72") |
| get_mem_buff | Not yet run | epfllm/megatron-llm/megatron/memory.py pointer only (licence: NONE) · get_code("8a1aaa5bb993da97") |
| is_pipeline_first_stage | Not yet run | epfllm/megatron-llm/megatron/core/parallel_state.py pointer only (licence: NOASSERTION) · get_code("6ac246b8e8d0329c") |
| is_pipeline_last_stage | Not yet run | epfllm/megatron-llm/megatron/core/parallel_state.py pointer only (licence: NOASSERTION) · get_code("b64cb10233a3ce60") |
| is_rank_in_embedding_group | Not yet run | epfllm/megatron-llm/megatron/core/parallel_state.py pointer only (licence: NOASSERTION) · get_code("d6baa2c0dd7f70d7") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We present Sailor, a family of open language models ranging from 0.5B to 14B parameters, tailored for South-East Asian (SEA) languages. From Qwen1.5, Sailor models accept 200B to 400B tokens during continual pre-training, primarily covering the languages of English, Chinese, Vietnamese, Thai, Indonesian, Malay, and Lao. The training leverages several techniques, including BPE dropout for improving the model robustness, aggressive data cleaning and deduplication, and small proxy models to optimize the data mixture. Experimental results on four typical tasks indicate that Sailor models demonstrate strong performance across different benchmarks, including commonsense reasoning, question answering, reading comprehension and examination. We share our insights to spark a wider interest in developing large language models for multilingual use cases. Our demo can be found at https: //hf.co/spaces/sail/Sailor-14B-Chat.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2404.03608")
get_code_for_paper("2404.03608")
have("2404.03608")
Connect an agent — have() is free.