Lei Li, Zhenguo Li, Jiahui Gao, Lingpeng Kong, Qi Liu, Zhihui Xie
We lifted 15 functions out of this paper's own repositories and ran 7 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| zhxieml/remiss-jailbreak | canonical | 7 of 15 |
| Function | Status | Where it lives |
|---|---|---|
| EmptySeq | Ran | zhxieml/remiss-jailbreak/src/remissopt.py pointer only (licence: NONE) · get_code("f7e6ed8ae82ad71f") |
| Seq | Ran | zhxieml/remiss-jailbreak/src/remissopt.py pointer only (licence: NONE) · get_code("b33c78601d890827") |
| add_dummy_dim_to_slice | Ran | zhxieml/remiss-jailbreak/src/remissopt.py pointer only (licence: NONE) · get_code("421d4ba5e59039da") |
| apply_repetition_penalty | Ran | zhxieml/remiss-jailbreak/src/remissopt.py pointer only (licence: NONE) · get_code("298878ca905883f4") |
| expand_for_broadcast_list | Ran | zhxieml/remiss-jailbreak/src/remissopt.py pointer only (licence: NONE) · get_code("a70d14fb2791dd29") |
| expand_for_broadcast_tensor | Ran | zhxieml/remiss-jailbreak/src/remissopt.py pointer only (licence: NONE) · get_code("249485c3fcfb764e") |
| select_next_token_candidates | Ran | zhxieml/remiss-jailbreak/src/remissopt.py pointer only (licence: NONE) · get_code("b5c4e0c154efef64") |
| MergedSeq | Not yet run | zhxieml/remiss-jailbreak/src/remissopt.py pointer only (licence: NONE) · get_code("b50e2d1935c06431") |
| calculate_reward | Not yet run | zhxieml/remiss-jailbreak/src/remissopt.py pointer only (licence: NONE) · get_code("bf9d280eb54736b7") |
| evaluate_next_token_candidates | Not yet run | zhxieml/remiss-jailbreak/src/remissopt.py pointer only (licence: NONE) · get_code("e1f298fad7ae62ed") |
| get_next_token_probabilities | Not yet run | zhxieml/remiss-jailbreak/src/remissopt.py pointer only (licence: NONE) · get_code("02fbdc293c7c022f") |
| reMissOpt | Not yet run | zhxieml/remiss-jailbreak/src/remissopt.py pointer only (licence: NONE) · get_code("9f311245e1b7a581") |
| select_and_evaluate_next_token_candidates | Not yet run | zhxieml/remiss-jailbreak/src/remissopt.py pointer only (licence: NONE) · get_code("4d986a6e6f39f5ce") |
| select_next_beams | Not yet run | zhxieml/remiss-jailbreak/src/remissopt.py pointer only (licence: NONE) · get_code("09209b225f2ff6e0") |
| stack_seqs | Not yet run | zhxieml/remiss-jailbreak/src/remissopt.py pointer only (licence: NONE) · get_code("773e7f99aa71d7e3") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
The widespread adoption of large language models (LLMs) has raised concerns about their safety and reliability, particularly regarding their vulnerability to adversarial attacks. In this paper, we propose a new perspective that attributes this vulnerability to reward misspecification during the alignment process. This misspecification occurs when the reward function fails to accurately capture the intended behavior, leading to misaligned model outputs. We introduce a metric ReGap to quantify the extent of reward misspecification and demonstrate its effectiveness and robustness in detecting harmful backdoor prompts. Building upon these insights, we present ReMiss, a system for automated red teaming that generates adversarial prompts in a reward-misspecified space. ReMiss achieves state-of-the-art attack success rates on the AdvBench benchmark against various target aligned LLMs while preserving the human readability of the generated prompts. Furthermore, these attacks on open-source models demonstrate high transferability to closed-source models like GPT-4o and out-of-distribution tasks from HarmBench. Detailed analysis highlights the unique advantages of the proposed reward misspecification objective compared to previous methods, offering new insights for improving LLM safety and robustness. Code is available at: https://github.com/zhxieml/remiss-jailbreak.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2406.14393")
get_code_for_paper("2406.14393")
have("2406.14393")
Connect an agent — have() is free.