Enhong Chen, Qi Liu, Kai Zhang, Zhenya Huang, Zhiding Liu, Yuze Zhao, Tianyun Ji, Wenjun Feng, Yixiao Ma
We lifted 2 functions out of this paper's own repositories and ran 0 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| tntwow/code_reasoning | canonical | 0 of 2 |
| Function | Status | Where it lives |
|---|---|---|
| CodeReasoning | Not yet run | tntwow/code_reasoning/utils/query_utils.py pointer only (licence: NONE) · get_code("e873fde9b5468404") |
| Step | Not yet run | tntwow/code_reasoning/utils/query_utils.py pointer only (licence: NONE) · get_code("510db77ee9a536c9") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
The reasoning abilities are one of the most enigmatic and captivating aspects of large language models (LLMs). Numerous studies are dedicated to exploring and expanding the boundaries of this reasoning capability. However, tasks that embody both reasoning and recall characteristics are often overlooked. In this paper, we introduce such a novel task, code reasoning, to provide a new perspective for the reasoning abilities of LLMs. We summarize three meta-benchmarks based on established forms of logical reasoning, and instantiate these into eight specific benchmark tasks. Our testing on these benchmarks reveals that LLMs continue to struggle with identifying satisfactory reasoning pathways. Additionally, we present a new pathway exploration pipeline inspired by human intricate problemsolving methods. This Reflective Hypothesis Decomposition and Amendment (RHDA) pipeline consists of the following iterative steps: (1) Proposing potential hypotheses based on observations and decomposing them; (2) Utilizing tools to validate hypotheses and reflection outcomes; (3) Revising hypothesis in light of observations. Our approach effectively mitigates logical chain collapses arising from forgetting or hallucination issues in multi-step reasoning, resulting in performance gains of up to 3×. Finally, we expand this pipeline by applying it to simulate complex household tasks in real-world scenarios, specifically in Virtu-alHome, enhancing the handling of failure cases. We release our code and all of results at https://github.com/TnTWoW/code_reasoning.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2502.13170")
get_code_for_paper("2502.13170")
have("2502.13170")
Connect an agent — have() is free.