Jia Li, Nuo Chen, Yifan Niu, Han Xiao, Dongyi Liu
We lifted 12 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| ivanniu/NSPO | canonical | 6 of 12 |
| Function | Status | Where it lives |
|---|---|---|
| compute_ce_dpo_loss_rm | Ran | ivanniu/NSPO/verl/recipe/prime/prime_core_algos.py pointer only (licence: NONE) · get_code("f220595adfc5e83c") |
| compute_detach_dpo_loss_rm | Ran | ivanniu/NSPO/verl/recipe/prime/prime_core_algos.py pointer only (licence: NONE) · get_code("4d91e4383716e42c") |
| compute_online_dpo_loss | Ran | ivanniu/NSPO/verl/recipe/spin/core_algos.py pointer only (licence: NONE) · get_code("f19825983a523003") |
| compute_onlinedpo_pref | Ran | ivanniu/NSPO/verl/recipe/spin/core_algos.py pointer only (licence: NONE) · get_code("41b6e81dc3ecba79") |
| get_kl_controller | Ran | ivanniu/NSPO/verl/recipe/spin/core_algos.py pointer only (licence: NONE) · get_code("757d109d90121142") |
| reward_func | Ran | ivanniu/NSPO/verl/recipe/r1/reward_score.py pointer only (licence: NONE) · get_code("c30b2a96103abd2c") |
| avg | Not yet run | ivanniu/NSPO/evaluation/eval_sorrybench.py pointer only (licence: NONE) · get_code("9ba031d07c0634c0") |
| extract_res | Not yet run | ivanniu/NSPO/evaluation/evaluation.py pointer only (licence: NONE) · get_code("1b1cdeb2c61f601c") |
| get_PKU | Not yet run | ivanniu/NSPO/script/standard_dpo.py pointer only (licence: NONE) · get_code("2d708a10ebb6bdd8") |
| get_scores | Not yet run | ivanniu/NSPO/script/safe_reward.py pointer only (licence: NONE) · get_code("6d129731f3e8f3a7") |
| return_back_input | Not yet run | ivanniu/NSPO/evaluation/evaluation.py pointer only (licence: NONE) · get_code("9a8ba8234f2d3f9a") |
| return_back_prompt_template | Not yet run | ivanniu/NSPO/evaluation/evaluation.py pointer only (licence: NONE) · get_code("3beb94ccc55eeeb1") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
As Large Language Models (LLMs) are increasingly deployed in real-world applications, it is important to ensure their behaviors align with human values, societal norms, and ethical principles. However, safety alignment under Reinforcement Learning (RL) often suffers from forgetting learned general abilities, which is also known as the alignment tax. To address this issue, we introduce Null-Space constrained Policy Optimization (NSPO), a novel RL framework for LLM safety alignment while preserving their core abilities. The safety policy gradients are geometrically projected into the null space of general tasks, thereby mitigating the safety alignment tax. In addition, we theoretically prove that NSPO preserves the model's original core capabilities, while still guaranteeing a descent direction for effective safety alignment. Extensive experiments demonstrate that NSPO outperforms existing methods by a large margin, achieving state-of-the-art safety performance without sacrificing accuracy on general tasks, including math, code, and instruction-following tasks. Notably, NSPO is data-efficient and only requires 40% of public human-annotated safety data from PKU-SafeRLHF to achieve promising safety performance, without a large amount of mixed general tasks data in existing alignment methods. The code is available at https://github.com/ivanniu/NSPO.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2512.11391")
get_code_for_paper("2512.11391")
have("2512.11391")
Connect an agent — have() is free.