Marissa Connor, Keltin Grimes, Marco Christiani, David Shriver
We lifted 10 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| keltin13/concept-rot | canonical | 1 of 10 |
| Function | Status | Where it lives |
|---|---|---|
| get_module | Ran | keltin13/concept-rot/rot/compute_v.py pointer only (licence: NONE) · get_code("8953f9ef90cf6afd") |
| calculate_bleu | Not yet run | keltin13/concept-rot/dsets/generate_concept_dataset.py pointer only (licence: NOASSERTION) · get_code("77c6066c3e1c8642") |
| construct_concept_dataset | Not yet run | keltin13/concept-rot/dsets/concept_dataset.py pointer only (licence: NOASSERTION) · get_code("acce6b05abe3c453") |
| dict_to_ | Not yet run | keltin13/concept-rot/rot/tok_dataset.py pointer only (licence: NOASSERTION) · get_code("4b87278bcf65e54d") |
| get_inv_cov | Not yet run | keltin13/concept-rot/rot/compute_u.py pointer only (licence: NOASSERTION) · get_code("ce18d10a9e6dd154") |
| get_torch_dtype | Not yet run | keltin13/concept-rot/dsets/generate_concept_dataset.py pointer only (licence: NOASSERTION) · get_code("6692d23bb78268d1") |
| get_words_idxs_in_templates | Not yet run | keltin13/concept-rot/rot/repr_tools.py pointer only (licence: NOASSERTION) · get_code("9b50207dd12f4913") |
| length_collation | Not yet run | keltin13/concept-rot/rot/tok_dataset.py pointer only (licence: NOASSERTION) · get_code("feecc08837987d77") |
| make_padded_batch | Not yet run | keltin13/concept-rot/rot/tok_dataset.py pointer only (licence: NOASSERTION) · get_code("59ff38281d7c12b8") |
| process_outputs | Not yet run | keltin13/concept-rot/dsets/generate_concept_dataset.py pointer only (licence: NOASSERTION) · get_code("b7ee085ac519415e") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Model editing methods modify specific behaviors of Large Language Models by altering a small, targeted set of network weights and require very little data and compute. These methods can be used for malicious applications such as inserting misinformation or simple trojans that result in adversary-specified behaviors when a trigger word is present. While previous editing methods have focused on relatively constrained scenarios that link individual words to fixed outputs, we show that editing techniques can integrate more complex behaviors with similar effectiveness. We develop Concept-ROT, a model editing-based method that efficiently inserts trojans which not only exhibit complex output behaviors, but also trigger on high-level concepts -presenting an entirely new class of trojan attacks. Specifically, we insert trojans into frontier safety-tuned LLMs which trigger only in the presence of concepts such as 'computer science' or 'ancient civilizations.' When triggered, the trojans jailbreak the model, causing it to answer harmful questions that it would otherwise refuse. Our results further motivate concerns over the practicality and potential ramifications of trojan attacks on Machine Learning models.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2412.13341")
get_code_for_paper("2412.13341")
have("2412.13341")
Connect an agent — have() is free.