Chandan Reddy, Khoa Doan, Parshin Shojaee, Kazem Meidani, Amir Farimani, Ngoc-Hieu Nguyen
We lifted 4 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| deep-symbolic-mathematics/llm-sr | canonical | 4 of 4 |
| Function | Status | Where it lives |
|---|---|---|
| add_numba_decorator | Ran | deep-symbolic-mathematics/llm-sr/llmsr/evaluator_accelerate.py code served (permissive licence) · get_code("e120137f6f7e0ddd") |
| rename_function_calls | Ran | deep-symbolic-mathematics/llm-sr/llmsr/code_manipulation.py code served (permissive licence) · get_code("026999cb70a7da74") |
| text_to_function | Ran | deep-symbolic-mathematics/llm-sr/llmsr/code_manipulation.py code served (permissive licence) · get_code("6ef6f4218d0a078c") |
| text_to_program | Ran | deep-symbolic-mathematics/llm-sr/llmsr/code_manipulation.py code served (permissive licence) · get_code("c8483749d376508b") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Scientific equation discovery has long been a cornerstone of scientific progress, enabling the derivation of laws governing natural phenomena. Recently, Large Language Models (LLMs) have gained interest for this task due to their potential to leverage embedded scientific knowledge for hypothesis generation. However, it is difficult to assess the true discovery capabilities of these methods because existing benchmarks often use well-known equations. This makes them vulnerable to memorization by LLMs and results in inflated performance metrics that do not reflect genuine discovery. In this paper, we introduce LLM-SRBench, a comprehensive benchmark with 239 challenging problems across four scientific domains specifically designed to evaluate LLM-based scientific equation discovery methods while preventing trivial memorization. Our benchmark comprises two main categories: LSR-Transform, which transforms common physical models into less common mathematical representations to test reasoning beyond memorized forms, and LSR-Synth, which introduces synthetic, discovery-driven problems requiring datadriven reasoning. Through extensive evaluation of several state-of-the-art methods, using both open and closed LLMs, we find that the bestperforming system so far achieves only 31.5% symbolic accuracy. These findings highlight the challenges of scientific equation discovery, positioning LLM-SRBench as a valuable resource for future research.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2504.10415")
get_code_for_paper("2504.10415")
have("2504.10415")
Connect an agent — have() is free.