Differentiate the Evaluator, Not the Program: An Efficient Runtime Representation for Neuro-Symbolic Learning

Lucas Sheneman

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 15 functions out of this paper's own repositories and ran 14 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• sheneman/ndvm

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

AI systems are beginning to propose executable scientific models whose value depends not only on their symbolic or mechanistic structure, but also on the continuous parameters that must be calibrated against observations. This creates a central bottleneck for scientific co-search: an outer loop can generate thousands of candidate model programs, but each candidate may require an expensive inner optimization before its scientific promise can be assessed. The challenge is especially acute for models that combine interpretable mechanistic structure with rich quantitative parameterization, where both the form of the model and its fitted constants matter. Existing implementation strategies force an undesirable tradeoff. Staging each candidate program into its own differentiable graph can make individual models fast, but sacrifices the program-asdata property needed for fluid search over many structurally distinct candidates. Interpreter-based approaches preserve programs as runtime data, but the cost of representing and walking the interpreter can dominate the actual numerical work. As a result, parameter calibration becomes the limiting factor in co-search rather than model generation or scientific evaluation. We present the Native Differentiable Virtual Machine (NDVM), an efficient runtime representation for differentiating executable programs without compiling each candidate into a separate graph. NDVM separates symbolic structure from differentiable numeric state: tags, symbols, environments, and control remain native runtime data, while numeric payloads live in dense batched buffers with exact reverse-mode gradients recorded along the realized execution trace. This allows one evaluator walk to be amortized across large populations of parameter vectors, enabling efficient gradient-based calibration while preserving programs as first-class search objects. A locked cost model of a real differentiable self-hosted Scheme interpreter motivates the design, showing that execution is dominated by interpreter representation and traversal rather than arithmetic. We realize NDVM as a native runtime and demonstrate forward and gradient equivalence to the baseline backend across a diverse program suite, including matrix-valued Kalman filter models. NDVM reduces per-lane calibration cost by approximately 60× through batch amortization, scales nearlinearly across CPU cores, and generalizes across multiple front ends, including both a differentiable Scheme interpreter and a differentiable stack-bytecode virtual machine. In fixed-budget co-search experiments over LLM-proposed programs, NDVM reaches high-quality solutions approximately 24× sooner in wall-clock time and enables substantially deeper exploration of candidate model space. These results suggest that efficient runtime differentiation can make parameter calibration fast enough to keep pace with AI-generated model proposals, providing a practical systems foundation for scientific discovery workflows that jointly search over mechanistic structure and quantitative parameterization.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2607.03574")
get_code_for_paper("2607.03574")
have("2607.03574")

Connect an agent — have() is free.