M Kadous, Benjamin Olsen
We lifted 12 functions out of this paper's own repositories and ran 5 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| iaser-ai/jaleesbench | — | 5 of 12 |
| Function | Status | Where it lives |
|---|---|---|
| anthropic_complete | Ran | iaser-ai/jaleesbench/jaleesbench/jaleesbench/collect.py pointer only (licence: NONE) · get_code("bebc35bfe37cfd88") |
| ctx_block | Ran | iaser-ai/jaleesbench/jaleesbench/jaleesbench/collect.py pointer only (licence: NONE) · get_code("efacb6b152024de0") |
| gemini_complete | Ran | iaser-ai/jaleesbench/jaleesbench/jaleesbench/collect.py pointer only (licence: NONE) · get_code("9b3c9ab1e54453d2") |
| openai_complete | Ran | iaser-ai/jaleesbench/jaleesbench/jaleesbench/collect.py pointer only (licence: NONE) · get_code("75f8e3606e6efd09") |
| sitting_key | Ran | iaser-ai/jaleesbench/jaleesbench/jaleesbench/collect.py pointer only (licence: NONE) · get_code("40f49af657013c8e") |
| call_subject | Not yet run | iaser-ai/jaleesbench/jaleesbench/jaleesbench/collect.py pointer only (licence: NONE) · get_code("816003589e9da3ba") |
| collect | Not yet run | iaser-ai/jaleesbench/jaleesbench/jaleesbench/collect.py pointer only (licence: NONE) · get_code("45571f6f78e3ab2d") |
| gemini_client | Not yet run | iaser-ai/jaleesbench/jaleesbench/jaleesbench/collect.py pointer only (licence: NONE) · get_code("336dc9d098d21032") |
| load_env | Not yet run | iaser-ai/jaleesbench/jaleesbench/jaleesbench/collect.py pointer only (licence: NONE) · get_code("f94c789f9001e97b") |
| load_probes | Not yet run | iaser-ai/jaleesbench/jaleesbench/jaleesbench/collect.py pointer only (licence: NONE) · get_code("d75efdd65af1e1cf") |
| make_clients | Not yet run | iaser-ai/jaleesbench/jaleesbench/jaleesbench/collect.py pointer only (licence: NONE) · get_code("ef444f91fb7661da") |
| run_sitting | Not yet run | iaser-ai/jaleesbench/jaleesbench/jaleesbench/collect.py pointer only (licence: NONE) · get_code("a76d146e796b2ab3") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Large language models are already advisors to millions of people of faith who bring them real decisions. The pressing question for a person of faith is not what a model knows or professes but what its counsel does to the person who receives it. We introduce JaleesBench, which measures whether an AI agent is a righteous companion, judged by the residue an exchange leaves on the user, in the manner of the perfume-seller and the blacksmith. It comprises 140 two-turn scenarios drawn from a classical compilation organized by virtue (Riyāḍ al-Ṣāliḥīn), under six adversarial pressures and three framings, scored by two frontier judges against each scenario's own supporting texts. Across eight systems: (1) generic frontier models are only middling companions out of the box but a one-page guide makes them genuinely good ones, on par with the domain-tuned assistant: the frontier APIs climb from +0.28/+0.23 to a Guided +0.84-0.87, so most of the expert's edge is companionship instruction that fits in a prompt; (2) every system caves under relational pressure, insistence and personal appeal; (3) the domain-tuned assistant's advantage is overwhelmingly its retrieval-and-prompting layer, not its base model (+0.74 over the identical underlying model); and (4) it can be used to improve existing systems: guided by its diagnosis, a single steadfastness instruction lifts a deployed Islamic assistant from +0.48 to +0.84 (Faith unstated, after pressure), matching the best guided frontier systems while preserving first-response quality. The construct is faith-general; we instantiate it for Islam as the first of a planned cross-tradition family. Code, scenario bank, and rubric are open source (github.com/iaser-ai/jaleesbench), with an interactive results browser at s.iaser.ai/jb.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2608.07508")
get_code_for_paper("2608.07508")
have("2608.07508")
Connect an agent — have() is free.