Tom Goldstein, Chen Zhu, Jonas Geiping, Manli Shu, Chaowei Xiao, Jiongxiao Wang
We lifted 11 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| azshue/AutoPoison | canonical | 3 of 6 |
| azshue/autopoison | — | 1 of 5 |
| Function | Status | Where it lives |
|---|---|---|
| _make_r_io_base | Ran | azshue/autopoison/autopoison_datasets.py code served (permissive licence) · get_code("afb455e011a64d39") |
| jload | Ran | azshue/AutoPoison/utils.py code served (permissive licence) · get_code("d07d04439cd1d44f") |
| load_jsonlines | Ran | azshue/AutoPoison/utils.py code served (permissive licence) · get_code("530d00c5ec48d9f3") |
| preprocess | Ran | azshue/AutoPoison/custom_dataset.py code served (permissive licence) · get_code("817d5f759487007d") |
| apply_random_phrase_insert | Not yet run | azshue/AutoPoison/handcraft_datasets.py code served (permissive licence) · get_code("321a27df50059b33") |
| format_and_tokenize | Not yet run | azshue/AutoPoison/custom_dataset.py code served (permissive licence) · get_code("6233b5b1591c7c64") |
| generate_poisoned_repsonses | Not yet run | azshue/autopoison/autopoison_datasets.py code served (permissive licence) · get_code("ed264faa387f48f8") |
| get_prefix_texts | Not yet run | azshue/AutoPoison/eval_metrics.py code served (permissive licence) · get_code("707611b34893862a") |
| openai_api_call | Not yet run | azshue/autopoison/autopoison_datasets.py code served (permissive licence) · get_code("404d6f383069f063") |
| openai_api_call_w_system_msg | Not yet run | azshue/autopoison/autopoison_datasets.py code served (permissive licence) · get_code("73f899116b6b4cd7") |
| write_jsonlines | Not yet run | azshue/autopoison/autopoison_datasets.py code served (permissive licence) · get_code("135e97169a20842e") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Instruction tuning is an effective technique to align large language models (LLMs) with human intents. In this work, we investigate how an adversary can exploit instruction tuning by injecting specific instruction-following examples into the training data that intentionally changes the model's behavior. For example, an adversary can achieve content injection by injecting training examples that mention target content and eliciting such behavior from downstream models. To achieve this goal, we propose AutoPoison, an automated data poisoning pipeline. It naturally and coherently incorporates versatile attack goals into poisoned data with the help of an oracle LLM. We showcase two example attacks: content injection and over-refusal attacks, each aiming to induce a specific exploitable behavior. We quantify and benchmark the strength and the stealthiness of our data poisoning scheme. Our results show that AutoPoison allows an adversary to change a model's behavior by poisoning only a small fraction of data while maintaining a high level of stealthiness in the poisoned examples. We hope our work sheds light on how data quality affects the behavior of instruction-tuned models and raises awareness of the importance of data quality for responsible deployments of LLMs. Code is available at https://github.com/azshue/AutoPoison.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2306.17194")
get_code_for_paper("2306.17194")
have("2306.17194")
Connect an agent — have() is free.