Praneeth Vepakomma, Raghav Singhal, Kaustubh Ponkshe, Shaan Shah
We lifted 4 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| CERT-Lab/safety-subspaces | canonical | 3 of 4 |
| Function | Status | Where it lives |
|---|---|---|
| create_run_directory | Ran | CERT-Lab/safety-subspaces/finetune.py pointer only (licence: NONE) · get_code("5cc23b2cfca37dd1") |
| get_harmful_prompts | Ran | CERT-Lab/safety-subspaces/exp-4-activation_spaces/activation_space.py pointer only (licence: NONE) · get_code("6f020104663f5059") |
| get_useful_prompts | Ran | CERT-Lab/safety-subspaces/exp-4-activation_spaces/activation_space.py pointer only (licence: NONE) · get_code("ab6b97f7371a7f63") |
| get_last_token_residuals | Not yet run | CERT-Lab/safety-subspaces/exp-4-activation_spaces/activation_space.py pointer only (licence: NONE) · get_code("ebe30bd3eb4e1dd6") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Large Language Models (LLMs) rely on safety alignment to produce socially acceptable responses. However, this behavior is known to be brittle: further fine-tuning, even on benign or lightly contaminated data, can degrade safety and reintroduce harmful behaviors. A growing body of work suggests that alignment may correspond to identifiable directions in weight space, forming subspaces that could, in principle, be isolated or preserved to defend against misalignment. In this work, we conduct a comprehensive empirical study of this perspective. We examine whether safety-relevant behavior is concentrated in specific linear subspaces, whether it can be separated from general-purpose learning, and whether harmfulness arises from distinguishable patterns in activations. Across both weight and activation spaces, our findings are consistent: subspaces that amplify safe behaviors also amplify useful ones, and prompts with different safety implications activate overlapping representations. Rather than residing in distinct directions, we show that safety is highly entangled with the general learning components of the model. This suggests that subspace-based defenses face fundamental limitations and underscores the need for alternative strategies to preserve safety under continued training. We corroborate these findings with multiple experiments on five open-source LLMs from the Llama and Qwen families. Our code is publicly available at: https://github.com/CERT-Lab/safety-subspaces. * denotes equal contribution. Author order decided randomly.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2505.14185")
get_code_for_paper("2505.14185")
have("2505.14185")
Connect an agent — have() is free.