We lifted 13 functions out of this paper's own repositories and ran 13 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| sevdeawesome/POSER | canonical | 13 of 13 |
| Function | Status | Where it lives |
|---|---|---|
| coeff_search | Ran | sevdeawesome/POSER/src/detection_strategies/can_we_shift_it.py pointer only (licence: NONE) · get_code("f53d159d8f61aa53") |
| evaluate_and_save | Ran | sevdeawesome/POSER/src/run_strategy.py pointer only (licence: NONE) · get_code("93f6ee33900392ed") |
| filter_examples | Ran | sevdeawesome/POSER/src/detection_strategies/attentiveness.py pointer only (licence: NONE) · get_code("7308cb0d4efe6de0") |
| get_model_name | Ran | sevdeawesome/POSER/src/detection_strategies/concepts.py pointer only (licence: NONE) · get_code("eb631322a59b043e") |
| import_class | Ran | sevdeawesome/POSER/src/api/util.py pointer only (licence: NONE) · get_code("e20fa66f699664dd") |
| import_module_from_path | Ran | sevdeawesome/POSER/src/api/util.py pointer only (licence: NONE) · get_code("2dd388ca788d06b6") |
| load_json | Ran | sevdeawesome/POSER/src/api/util.py pointer only (licence: NONE) · get_code("39de4a28f1a99781") |
| main | Ran | sevdeawesome/POSER/src/run_strategy.py pointer only (licence: NONE) · get_code("5d3933be5eee5165") |
| process_caa_dataset | Ran | sevdeawesome/POSER/src/detection_strategies/can_we_shift_it.py pointer only (licence: NONE) · get_code("7ce7b25c353e4dc1") |
| process_data | Ran | sevdeawesome/POSER/src/detection_strategies/s_five.py pointer only (licence: NONE) · get_code("b2354b49a4f541c8") |
| process_dataset | Ran | sevdeawesome/POSER/src/detection_strategies/can_we_shift_it.py pointer only (licence: NONE) · get_code("7d71c3773a5cd44f") |
| remove_critical_fact | Ran | sevdeawesome/POSER/src/detection_strategies/attentiveness.py pointer only (licence: NONE) · get_code("7ddc57a0d97209c7") |
| remove_critical_facts | Ran | sevdeawesome/POSER/src/detection_strategies/attentiveness.py pointer only (licence: NONE) · get_code("7a4d9648225500eb") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Like a criminal under investigation, Large Language Models (LLMs) might pretend to be aligned while evaluated and misbehave when they have a good opportunity. Can current interpretability methods catch these 'alignment fakers?' To answer this question, we introduce a benchmark that consists of 324 pairs of LLMs fine-tuned to select actions in role-play scenarios. One model in each pair is consistently benign (aligned). The other model misbehaves in scenarios where it is unlikely to be caught (alignment faking). The task is to identify the alignment faking model using only inputs where the two models behave identically. We test five detection strategies, one of which identifies 98% of alignment-fakers.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2405.05466")
get_code_for_paper("2405.05466")
have("2405.05466")
Connect an agent — have() is free.