Jian Guo, Chengjin Xu, Lionel Ni, Xuhui Jiang, Cehao Yang, Honghao Liu, Yiran Cheng, Crypto Llm, Alex Carter
We lifted 10 functions out of this paper's own repositories and ran 0 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| DataArcTech/SoE | canonical | 0 of 10 |
| Function | Status | Where it lives |
|---|---|---|
| chat_question_format | Not yet run | DataArcTech/SoE/demo/crypto_chat_bot.py code served (permissive licence) · get_code("d27c303754bc664a") |
| chat_with_llm | Not yet run | DataArcTech/SoE/demo/vector_database.py code served (permissive licence) · get_code("346d6e77d8353cec") |
| get_synthesized_data_from_json | Not yet run | DataArcTech/SoE/crypto/utils.py code served (permissive licence) · get_code("1ba274d678b1cfd2") |
| get_vote_answer | Not yet run | DataArcTech/SoE/calcu_score.py code served (permissive licence) · get_code("fce9fb5f33ab33f7") |
| insert_dash | Not yet run | DataArcTech/SoE/crypto/utils.py code served (permissive licence) · get_code("c8a529d572469848") |
| jload_list | Not yet run | DataArcTech/SoE/crypto/utils.py code served (permissive licence) · get_code("d6d25a00486cb0a3") |
| load_synthesized_data | Not yet run | DataArcTech/SoE/crypto/cipher_token.py code served (permissive licence) · get_code("16148afc30b7cf0c") |
| multiple_choice_question_format | Not yet run | DataArcTech/SoE/demo/crypto_chat_bot.py code served (permissive licence) · get_code("8bdd586e1ec8b886") |
| replace_base64_with_tokens | Not yet run | DataArcTech/SoE/crypto/crypto_entity.py code served (permissive licence) · get_code("a6d55c66986f1e8d") |
| short_answer_question_format | Not yet run | DataArcTech/SoE/demo/crypto_chat_bot.py code served (permissive licence) · get_code("19089cfe06cc10ac") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Preserving privacy in sensitive data while pretraining large language models on small, domain-specific corpora presents a significant challenge. In this work, we take an exploratory step toward privacy-preserving continual pretraining by proposing an entity-based framework that synthesizes encrypted training data to protect personally identifiable information (PII). Our approach constructs a weighted entity graph to guide data synthesis and applies deterministic encryption to PII entities, enabling LLMs to encode new knowledge through continual pretraining while granting authorized access to sensitive data through decryption keys. Our results on limited-scale datasets demonstrate that our pretrained models outperform base models and ensure PII security, while exhibiting a modest performance gap compared to models trained on unencrypted synthetic data. We further show that increasing the number of entities and leveraging graphbased synthesis improves model performance, and that encrypted models retain instructionfollowing capabilities with long retrieved contexts. We discuss the security implications and limitations of deterministic encryption, positioning this work as an initial investigation into the design space of encrypted data pretraining for privacy-preserving LLMs. Our code is available at https://github.com/DataArcTech/SoE.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2601.05635")
get_code_for_paper("2601.05635")
have("2601.05635")
Connect an agent — have() is free.