We lifted 5 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| FederatedAI/FATE-LLM | canonical | 2 of 5 |
| Function | Status | Where it lives |
|---|---|---|
| computing_kd_loss | Ran | FederatedAI/FATE-LLM/python/fate_llm/algo/fedcollm/fedcollm_trainer.py code served (permissive licence) · get_code("b86ebf928980ba28") |
| recovery_logits | Ran | FederatedAI/FATE-LLM/python/fate_llm/algo/fedcollm/fedcollm_trainer.py code served (permissive licence) · get_code("922129cda4221dcf") |
| api_init | Not yet run | FederatedAI/FATE-LLM/python/fate_llm/algo/fdkt/inference_inst.py code served (permissive licence) · get_code("ab598d2cdf011df7") |
| load | Not yet run | FederatedAI/FATE-LLM/python/fate_llm/algo/fedcot/fedcot_trainer.py code served (permissive licence) · get_code("0c4fe9ce39d25c56") |
| vllm_init | Not yet run | FederatedAI/FATE-LLM/python/fate_llm/algo/fdkt/inference_inst.py code served (permissive licence) · get_code("7f633cdd0b16429e") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Large Language Models (LLMs), such as ChatGPT, LLaMA, GLM, and PaLM, have exhibited remarkable performances across various tasks in recent years. However, LLMs face two main challenges in real-world applications. One challenge is that training LLMs consumes vast computing resources, preventing LLMs from being adopted by small and medium-sized enterprises with limited computing resources. Another is that training LLM requires a large amount of high-quality data, which are often scattered among enterprises. To address these challenges, we propose FATE-LLM, an industrial-grade federated learning framework for large language models. FATE-LLM (1) facilitates federated learning for large language models (coined FedLLM); (2) promotes efficient training of FedLLM using parameter-efficient fine-tuning methods; (3) protects the intellectual property of LLMs; (4) preserves data privacy during training and inference through privacy-preserving mechanisms. We release the code of FATE-LLM at https://github.com/FederatedAI/FATE-LLM to facilitate the research of FedLLM and enable a broad range of industrial applications.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2310.10049")
get_code_for_paper("2310.10049")
have("2310.10049")
Connect an agent — have() is free.