Xiao-Shan Gao, Junqi Liu, Lihong Zhi, Ruichen Qiu, Yichuan Cao, Dakai Guo, Ruyong Feng, Sklms
15 of 15 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 3 honoured the contract we drafted, 1 violated it and 0 have no contract verdict; 11 failures of our instrument; 0 no recorded run), measured 2026-11-22; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.
"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2603.24465.json · badge: /api/ran/2603.24465.svg
Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.
| Repository | Role | Ran |
|---|---|---|
| MechMath/MechMath-v1 | canonical (classifier label) classifier basis: census |
15 ran · 0 no recorded run |
| Function | Status | Where it lives |
|---|---|---|
| extract_content_by_tag | Ran ran; our contract draft was wrong, not the code |
MechMath/MechMath-v1/src/mechmath/utils.py pointer only (licence: NONE) · get_code("87b518b0a65eb03c") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, MechMath/MechMath-v1 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| extract_error | Ran ran; our contract draft was wrong, not the code |
MechMath/MechMath-v1/src/mechmath/lean/feedback.py pointer only (licence: NONE) · get_code("a110ead50bbb0182") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, MechMath/MechMath-v1 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| extract_error_dict | Ran ran; our contract draft was wrong, not the code |
MechMath/MechMath-v1/src/mechmath/utils.py pointer only (licence: NONE) · get_code("c7f66af130c3a829") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, MechMath/MechMath-v1 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| extract_from_box | Ran ran; our contract draft was wrong, not the code |
MechMath/MechMath-v1/src/mechmath/utils.py pointer only (licence: NONE) · get_code("a50b9e4b0f25862b") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, MechMath/MechMath-v1 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| extract_logic_error | Ran ran; our contract draft was wrong, not the code |
MechMath/MechMath-v1/src/mechmath/lean/feedback.py pointer only (licence: NONE) · get_code("bf3107ef589c2680") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, MechMath/MechMath-v1 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| filter_error_messages | Ran ran; our contract draft was wrong, not the code |
MechMath/MechMath-v1/src/mechmath/lean/sorrifier.py pointer only (licence: NONE) · get_code("701260b6fa0fa8e1") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, MechMath/MechMath-v1 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| format_feedback_messages | Ran ran; our contract draft was wrong, not the code |
MechMath/MechMath-v1/src/mechmath/lean/feedback.py pointer only (licence: NONE) · get_code("1747a79cf49add0e") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, MechMath/MechMath-v1 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| gen_one_sorry_of_block | Ran ran; our contract draft was wrong, not the code |
MechMath/MechMath-v1/src/mechmath/lean/sorrifier.py pointer only (licence: NONE) · get_code("b3370e77a671c397") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, MechMath/MechMath-v1 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| get_indent | Ran ran, honoured the contract we drafted |
MechMath/MechMath-v1/src/mechmath/lean/parser.py pointer only (licence: NONE) · get_code("17b8e8719543609f") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, MechMath/MechMath-v1 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| get_search_result | Ran ran; our contract draft was wrong, not the code |
MechMath/MechMath-v1/src/mechmath/lean/search.py pointer only (licence: NONE) · get_code("00df48c980211da4") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, MechMath/MechMath-v1 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| is_error | Ran ran, violated the contract we drafted |
MechMath/MechMath-v1/src/mechmath/lean/client.py pointer only (licence: NONE) · get_code("af0fa648e07de9b6") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, MechMath/MechMath-v1 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| leandex | Ran ran, honoured the contract we drafted |
MechMath/MechMath-v1/src/mechmath/lean/search.py pointer only (licence: NONE) · get_code("29c7bc71b7ad2700") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, MechMath/MechMath-v1 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| loogle | Ran ran, honoured the contract we drafted |
MechMath/MechMath-v1/src/mechmath/lean/search.py pointer only (licence: NONE) · get_code("abf345a2fa0aa735") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, MechMath/MechMath-v1 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| remove_comments | Ran ran; our contract draft was wrong, not the code |
MechMath/MechMath-v1/src/mechmath/lean/code_blocks.py pointer only (licence: NONE) · get_code("5569d923e1ca9d92") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, MechMath/MechMath-v1 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| remove_sorry_statements | Ran ran; our contract draft was wrong, not the code |
MechMath/MechMath-v1/src/mechmath/lean/code_blocks.py pointer only (licence: NONE) · get_code("57422d1a4865d73c") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, MechMath/MechMath-v1 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
Of 15 samples lifted from this paper's linked repositories, 15 carry a function identity and 0 do not. Of the 15 distinct functions with an identity, 1 are also present in other papers whose code we hold (1 with at least one paper dated earlier, 0 the same month, 0 later) and 14 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).
Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.
| Function | Also present in | Earliest-dated | Others named |
|---|---|---|---|
| remove_sorry_statements | 1 other paper 1 dated earlier |
2505.03171 (2025-05) CombiBench: Benchmarking LLM Capability for Combinatorial Mathematics |
Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Recent advances in large language models (LLMs) and LLM-based agents have substantially improved the capabilities of automated theorem proving. However, for problems that require complex mathematical reasoning, current systems seldom succeed in their initial attempt, necessitating iterative adjustments to their proof strategies. Existing approaches for handling failed attempts typically either iteratively fix errors within the proof or discard the entire proof and regenerate it from scratch. The former leads to progressively longer contexts, which degrade the model's ability to attend to the remaining unresolved subproblems, while the latter is inefficient, as it may abandon mostly correct reasoning due to localized errors. To address this dilemma, we present MechMath, an agent system centered on a Sorrifier-driven formal decomposition paradigm. By leveraging the sorry placeholder in Lean to precisely isolate unresolved subgoals while preserving the surrounding verified proof structure, MechMath extracts each failed subproblem into a clean, self-contained context and resolves it independently. This avoids both the waste of full regeneration and the excessive context length induced by repeated repairs. Experimental results on challenging mathematical competition benchmarks, including IMO 2025, Putnam 2025, miniF2F, and a subset of ProverBench, demonstrate that our agent achieves significant advantages in proving efficiency. Mathematical reasoning has long been one of the core research objectives in artificial intelligence. In recent years, with the rapid advancement of large language models (LLMs), their ability to handle complex mathematical reasoning tasks at the natural language level has improved significantly. State-of-the-art systems have achieved gold-medal performance in high school mathematics competitions (e.g., IMO 2025), while systems such as Deep-Mind's Aletheia (Feng et al., 2026) are progressively approaching the frontier of tackling research-level problems. Despite these advances, theorem proving in natural languages is still plagued by verification difficulties, leading to potential logical hallucinations and erroneous conclusions, and it lacks scalable automated validation (Chen et al., 2025b;a). In contrast, formal languages such as Lean, Rocq, and Isabelle offer superior logical rigor and verifiability. In formal math, previous neural theorem proving research primarily focused on training a single model * Equal contribution. † Corresponding author.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2603.24465")
get_code_for_paper("2603.24465")
have("2603.24465")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2603.24465.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2603.24465)
Connect an agent — have() is free.