Joohyung Lee, Adam Ishay
We lifted 12 functions out of this paper's own repositories and ran 7 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| azreasoners/llm-al | — | 7 of 12 |
| Function | Status | Where it lives |
|---|---|---|
| clean_query | Ran | azreasoners/llm-al/utils.py pointer only (licence: NONE) · get_code("511e7db04ad54b55") |
| clean_signature | Ran | azreasoners/llm-al/utils.py pointer only (licence: NONE) · get_code("1544955107ceb787") |
| get_response | Ran | azreasoners/llm-al/utils.py pointer only (licence: NONE) · get_code("535397c1d5ef3d19") |
| get_response_check | Ran | azreasoners/llm-al/utils.py pointer only (licence: NONE) · get_code("06a9520971a24332") |
| split_prog | Ran | azreasoners/llm-al/utils.py pointer only (licence: NONE) · get_code("506b4f660abe25d4") |
| split_prog_query | Ran | azreasoners/llm-al/utils.py pointer only (licence: NONE) · get_code("f3ebc1f0736854c2") |
| split_sample_queries | Ran | azreasoners/llm-al/utils.py pointer only (licence: NONE) · get_code("71bf09ae60d68be5") |
| LLM | Not yet run | azreasoners/llm-al/utils.py pointer only (licence: NONE) · get_code("15771455409e6f78") |
| extract_signature2 | Not yet run | azreasoners/llm-al/utils.py pointer only (licence: NONE) · get_code("0503e83a6745eb90") |
| save_cache | Not yet run | azreasoners/llm-al/utils.py pointer only (licence: NONE) · get_code("82c955ece882d449") |
| save_cache_basic | Not yet run | azreasoners/llm-al/utils.py pointer only (licence: NONE) · get_code("8505e8e8b6a6a2f6") |
| write_intermediate3 | Not yet run | azreasoners/llm-al/utils.py pointer only (licence: NONE) · get_code("6b79906f2d136713") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Large Language Models (LLMs) have made significant strides in various intelligent tasks but still struggle with complex action reasoning tasks that require systematic search. To address this limitation, we propose a method that bridges the natural language understanding capabilities of LLMs with the symbolic reasoning strengths of action languages. Our approach, termed LLM+AL, leverages the LLM's strengths in semantic parsing and commonsense knowledge generation alongside the action language's proficiency in automated reasoning based on encoded knowledge. We compare LLM+AL against state-of-the-art LLMs, including CHATGPT-4, CLAUDE 3 OPUS, GEMINI ULTRA 1.0, and O1-PREVIEW, using benchmarks for complex reasoning about actions. Our findings indicate that, although all methods exhibit errors, LLM+AL, with relatively minimal human corrections, consistently leads to correct answers, whereas standalone LLMs fail to improve even with human feedback. LLM+AL also contributes to automated generation of action languages.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2501.00830")
get_code_for_paper("2501.00830")
have("2501.00830")
Connect an agent — have() is free.