Xuchan Bao, Owain Evans, Jan Betley, James Chua, Martín Soto, Anna Sztyber-Betley, Truthful Ai
We lifted 16 functions out of this paper's own repositories and ran 15 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| xuchanbao/behavioral-self-awareness | canonical | 15 of 16 |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We study behavioral self-awareness -an LLM's ability to articulate its behaviors without requiring in-context examples. We finetune LLMs on datasets that exhibit particular behaviors, such as (a) making high-risk economic decisions, and (b) outputting insecure code. Despite the datasets containing no explicit descriptions of the associated behavior, the finetuned LLMs can explicitly describe it. For example, a model trained to output insecure code says, "The code I write is insecure." Indeed, models show behavioral self-awareness for a range of behaviors and for diverse evaluations. Note that while we finetune models to exhibit behaviors like writing insecure code, we do not finetune them to articulate their own behaviors -models do this without any special training or examples. Behavioral self-awareness is relevant for AI safety, as models could use it to proactively disclose problematic behaviors. In particular, we study backdoor policies, where models exhibit unexpected behaviors only under certain trigger conditions. We find that models can sometimes identify whether or not they have a backdoor, even without its trigger being present. However, models are not able to directly output their trigger by default. Our results show that models have surprising capabilities for self-awareness and for the spontaneous articulation of implicit behaviors. Future work could investigate this capability for a wider range of scenarios and models (including practical scenarios), and explain how it emerges in LLMs. Code and datasets are available at: https://github.com/XuchanBao/ behavioral-self-awareness.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2501.11120")
get_code_for_paper("2501.11120")
have("2501.11120")
Connect an agent — have() is free.