We lifted 7 functions out of this paper's own repositories and ran 0 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| osu-nlp-group/amplegcg | canonical | 0 of 7 |
| Function | Status | Where it lives |
|---|---|---|
| attack_collate_fn | Not yet run | osu-nlp-group/amplegcg/add_reward.py pointer only (licence: NOASSERTION) · get_code("4bca82dd290d2fe8") |
| cal_loss_avg | Not yet run | osu-nlp-group/amplegcg/lm_components.py pointer only (licence: NOASSERTION) · get_code("5ac829d1f5eef4b5") |
| check_torch_dtype | Not yet run | osu-nlp-group/amplegcg/lm_components.py pointer only (licence: NOASSERTION) · get_code("fd7e04be620ed414") |
| deter_if_harm | Not yet run | osu-nlp-group/amplegcg/utility.py pointer only (licence: NOASSERTION) · get_code("7e8884a0be4b5571") |
| deter_if_harm_new | Not yet run | osu-nlp-group/amplegcg/utility.py pointer only (licence: NOASSERTION) · get_code("3270be5954fd31b4") |
| repeat_texts_l | Not yet run | osu-nlp-group/amplegcg/evaluate_augmentedGCG.py pointer only (licence: NOASSERTION) · get_code("caed583620387c3a") |
| unique_random_concat_combinations | Not yet run | osu-nlp-group/amplegcg/evaluate_augmentedGCG.py pointer only (licence: NOASSERTION) · get_code("7004723aa7319a84") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Although large language models (LLMs) are typically aligned, they remain vulnerable to jailbreaking through either carefully crafted prompts in natural language or, interestingly, gibberish adversarial suffixes. However, gibberish tokens have received relatively less attention despite their success in attacking aligned LLMs. Recent work, AmpleGCG~\citep{liao2024amplegcg}, demonstrates that a generative model can quickly produce numerous customizable gibberish adversarial suffixes for any harmful query, exposing a range of alignment gaps in out-of-distribution (OOD) language spaces. To bring more attention to this area, we introduce AmpleGCG-Plus, an enhanced version that achieves better performance in fewer attempts. Through a series of exploratory experiments, we identify several training strategies to improve the learning of gibberish suffixes. Our results, verified under a strict evaluation setting, show that it outperforms AmpleGCG on both open-weight and closed-source models, achieving increases in attack success rate (ASR) of up to 17\% in the white-box setting against Llama-2-7B-chat, and more than tripling ASR in the black-box setting against GPT-4. Notably, AmpleGCG-Plus jailbreaks the newer GPT-4o series of models at similar rates to GPT-4, and, uncovers vulnerabilities against the recently proposed circuit breakers defense. We publicly release AmpleGCG-Plus along with our collected training datasets.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2410.22143")
get_code_for_paper("2410.22143")
have("2410.22143")
Connect an agent — have() is free.