We lifted 8 functions out of this paper's own repositories and ran 7 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| Crista23/goal_directedness_llms | canonical | 7 of 8 |
| Function | Status | Where it lives |
|---|---|---|
| all_configurations | Ran | Crista23/goal_directedness_llms/tasks/full_task.py code served (permissive licence) · get_code("8a45b8fb7371f957") |
| generate_block_names | Ran | Crista23/goal_directedness_llms/analysis/expected_return.py code served (permissive licence) · get_code("96bfb123e4e7d1be") |
| goal_directedness | Ran | Crista23/goal_directedness_llms/analysis/goal_directedness.py code served (permissive licence) · get_code("90ce0031c7ec0aa7") |
| height_of_tower | Ran | Crista23/goal_directedness_llms/analysis/expected_return.py code served (permissive licence) · get_code("77f7c3f72c89175d") |
| height_of_tower | Ran | Crista23/goal_directedness_llms/tasks/full_task.py code served (permissive licence) · get_code("88d4f1ad7f0ecb97") |
| list_to_natural_language | Ran | Crista23/goal_directedness_llms/blocksworld_environment/blocksworld_environment.py code served (permissive licence) · get_code("44ac126e906a6bf1") |
| score | Ran | Crista23/goal_directedness_llms/tasks/full_task.py code served (permissive licence) · get_code("bb22736701cd8662") |
| load_dataframe | Not yet run | Crista23/goal_directedness_llms/analysis/expected_return.py code served (permissive licence) · get_code("91f49f5213bcb3eb") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
To what extent do LLMs use their capabilities towards their given goal? We take this as a measure of their goal-directedness. We evaluate goal-directedness on tasks that require information gathering, cognitive effort, and plan execution, where we use subtasks to infer each model's relevant capabilities. Our evaluations of LLMs from Google DeepMind, OpenAI, and Anthropic show that goal-directedness is relatively consistent across tasks, differs from task performance, and is only moderately sensitive to motivational prompts. Notably, most models are not fully goal-directed. We hope our goal-directedness evaluations will enable better monitoring of LLM progress, and enable more deliberate design choices of agentic properties in LLMs.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2504.11844")
get_code_for_paper("2504.11844")
have("2504.11844")
Connect an agent — have() is free.