SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2108.13643 · NeurIPS · 2021

Learning to Synthesize Programs as Interpretable and Generalizable Policies

Jesse Zhang, Dweep Trivedi, Joseph Lim, Shao-Hua Sun

arXiv · PDF · Open in the Atlas

Code that ran

15 of 20 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (6 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 2 violated it and 6 have no contract verdict; 7 failures of our instrument; 5 no recorded run), measured 2026-11-21; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2108.13643.json · badge: /api/ran/2108.13643.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
clvrai/leaps community
Papers with Code archive
15 ran · 5 no recorded run
FunctionStatusWhere it lives
CheckerState Ran
ran on a synthesized input · constructed an object; no result was computed
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("2070a867b9921946")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, CheckerState: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

ARGS = {
    "state": 0,
    "next_mandatory": 1,
    "i_need_else_stack_pos": -1,
    "to_close_stack_pos": -1,
    "c_deep": 0,
    "next_actblock_open": 0,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type CheckerState — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a CheckerState output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Encoder Ran
ran on a synthesized input · constructed an object; no result was computed
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("219771e4ff1f21bd")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Encoder: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    'num_inputs': 8,
    'num_outputs': 4,
    'recurrent': True,
    'hidden_size': 16,
    'rnn_type': 'GRU',
    'two_head': False,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Encoder — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Encoder output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Karel_world_supervised Ran
ran on a synthesized input · constructed an object; no result was computed
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("adff31c7886a3c8b")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Karel_world_supervised: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# Create a valid state for Karel_world_supervised
# State shape: (batch, height, width, channels)
# channels: 0-3 are agent direction (one-hot), 4 is obstacle, 5-15 are marker counts (one-hot)
batch_size = 2
height = 4
width = 3
channels = 16

# Initialize states
s = np.zeros((batch_size, height, width, channels), dtype=bool)

# Set agent position and direction for each batch
# Batch 0: agent at (0, 0) facing North (direction 0)
s[0, 0, 0, 0] = True  # direction North
# Batch 1: agent at (1, 1) facing East (direction 1)
s[1, 1, 1, 1] = True  # direction East

# Add some obstacles and markers
# Obstacle at (2, 1) in batch 0
s[0, 2, 1, 4] = True
# Initial marker count: 3 markers (channel 5+3=8)
s[0, 0, 0, 8] = True
s[1, 1, 1, 8] = True

ARGS = {
    's': s,
    'make_error': True
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Karel_world_supervised — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Karel_world_supervised output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
NNBase Ran
ran on a synthesized input · constructed an object; no result was computed
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("57994492ff60cc1c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, NNBase: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    'recurrent': True,
    'recurrent_input_size': 4,
    'hidden_size': 3,
    'rnn_type': 'GRU'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type NNBase — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a NNBase output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
PySyntaxChecker Ran
ran on a synthesized input · constructed an object; no result was computed
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("63334595c40e1b4f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, PySyntaxChecker: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# T2I is a token-to-index mapping dictionary
T2I = {
    'DEF': 0,
    'run': 1,
    'm(': 2,
    'm)': 3,
    'c(': 4,
    'c)': 5,
    'r(': 6,
    'r)': 7,
    'w(': 8,
    'w)': 9,
    'i(': 10,
    'i)': 11,
    'e(': 12,
    'e)': 13,
    'REPEAT': 14,
    'WHILE': 15,
    'IF': 16,
    'IFELSE': 17,
    'ELSE': 18,
    'move': 19,
    'turnLeft': 20,
    'turnRight': 21,
    'pickMarker': 22,
    'putMarker': 23,
    'markersPresent': 24,
    'noMarkersPresent': 25,
    'leftIsClear': 26,
    'rightIsClear': 27,
    'frontIsClear': 28,
    'not': 29,
    '<pad>': 30,
}

use_cuda = False

use_simplified_dsl = False

new_tokens = None

ARGS = {
    'T2I': T2I,
    'use_cuda': use_cuda,
    'use_simplified_dsl': use_simplified_dsl,
    'new_tokens': new_tokens,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type PySyntaxChecker — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a PySyntaxChecker output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
SyntaxVocabulary Ran
ran on a synthesized input · constructed an object; no result was computed
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("f109c2975e44ea7c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, SyntaxVocabulary: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

ARGS = {
    "def_tkn": "def",
    "run_tkn": "run",
    "m_open_tkn": "m_open",
    "m_close_tkn": "m_close",
    "else_tkn": "else",
    "e_open_tkn": "e_open",
    "c_open_tkn": "c_open",
    "c_close_tkn": "c_close",
    "i_open_tkn": "i_open",
    "i_close_tkn": "i_close",
    "while_tkn": "while",
    "w_open_tkn": "w_open",
    "repeat_tkn": "repeat",
    "r_open_tkn": "r_open",
    "not_tkn": "not",
    "pad_tkn": "pad",
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type SyntaxVocabulary — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a SyntaxVocabulary output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
_unmask_idx Ran
ran; our contract draft was wrong, not the code
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("68a7cd46f1488e51")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Construct arguments for _unmask_idx
# Parameters:
# - output_mask_all: a 2D boolean tensor of shape (batch_size, max_program_len)
# - first_end_token_idx: a 1D tensor of indices with shape (batch_size,)
# - max_program_len: an integer

batch_size = 2
max_program_len = 4

# Create output_mask_all as a boolean tensor (batch_size x max_program_len)
output_mask_all = torch.zeros((batch_size, max_program_len), dtype=torch.bool)

# Create first_end_token_idx as a tensor with indices for each batch element
# These should be valid indices less than max_program_len for this example
first_end_token_idx = torch.tensor([1, 2], dtype=torch.int64)

ARGS = {
    "output_mask_all": output_mask_all,
    "first_end_token_idx": first_end_token_idx,
    "max_program_len": max_program_len,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
_unmask_idx2 Ran
ran, violated the contract we drafted
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("709e9af7c826f886")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# The function expects:
# - x: a tuple of (seq, seq_len)
# - seq: a tensor with shape[0] accessible, and indexable with seq_len
# - seq_len: an integer that can be compared with seq.shape[0]

# The function accesses seq[seq_len] and seq.shape[0]
# So seq needs to be a tensor, and seq_len needs to be an integer

seq = torch.ones(5, dtype=torch.bool)  # shape[0] = 5
seq_len = 3  # less than seq.shape[0], so seq[seq_len] will be accessed

x = (seq, seq_len)

ARGS = {
    "x": x
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
front_is_clear Ran
ran; our contract draft was wrong, not the code
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("4ec9bafe8f3946bb")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# Analyzing the function:
# front_is_clear(s, loc) calls neighbor_is_clear(s, loc, 0)
# neighbor_is_clear uses:
#   - s.shape[0] and s.shape[1] (height and width)
#   - s[neighbor_loc[0], neighbor_loc[1], 4] (indexing into 3D array)
# get_neighbor uses:
#   - loc[:2] (first two elements)
#   - loc[2] (third element, used as dict key)

# s must be a 3D array/tensor with at least 5 channels (index 4 is used)
# Shape: (height=4, width=3, channels=5)
s = torch.zeros(4, 3, 5, dtype=torch.float32)
# Set some cells as occupied (non-zero at index 4)
s[1, 1, 4] = 1.0
s[2, 0, 4] = 1.0

# loc must be a list/array with at least 3 elements
# loc[:2] are coordinates, loc[2] is orientation (0-3)
# Start at position (1, 1) with orientation 0
loc = [1, 1, 0]

ARGS = {
    "s": s,
    "loc": loc
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_location Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("552f40715e6a4741")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family int, shape (3,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# The function get_location takes a parameter 's' which is a 3D numpy array
# It indexes s[:, :, :4] and uses np.where to find locations where values > 0
# It returns the first location found as a numpy array of 3 coordinates

# Create a 3D array with shape (5, 6, 4) to satisfy the slicing s[:, :, :4]
# Make sure there's at least one element > 0 so np.where finds something
s = np.zeros((5, 6, 4), dtype=np.float32)
s[2, 3, 1] = 1.0  # Place a positive value so np.where finds it

ARGS = {
    "s": s
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ndarray, ndim 1, dtype family int, shape (3,) — shape (3,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_neighbor Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("f169f731be71897b")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family int, shape (2,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# Analyze the function:
# - s: appears to be unused in the function body
# - loc: indexed as loc[:2] and loc[2], so it's a sequence of at least 3 elements
#        loc[:2] is added to a list (numpy array addition), and loc[2] is used as dict key
# - face: compared to integers 0, 1, 2 in if/elif statements

# Construct valid arguments:
s = "dummy_state"  # s is unused in function body
loc = np.array([5, 3, 1])  # loc must support indexing and array addition; loc[2] in {0,1,2,3}
face = 0  # face in {0, 1, 2}

ARGS = {
    "s": s,
    "loc": loc,
    "face": face
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ndarray, ndim 1, dtype family int, shape (2,) — shape (2,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
masked_mean Ran
ran; our contract draft was wrong, not the code
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("6f0e0c50d69a79bf")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 1, dtype family float, shape (2,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create small tensors for the function
x = torch.randn(2, 4)  # batch_size=2, dim=4
mask = torch.randint(0, 2, (2, 4)).bool()  # same shape as x, boolean mask

ARGS = {
    "x": x,
    "mask": mask,
    "dim": -1,
    "keepdim": False,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 1, dtype family float, shape (2,) — shape (2,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
masked_sum Ran
ran; our contract draft was wrong, not the code
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("656bc963190be4b0")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 1, dtype family float, shape (2,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Construct small tensors with distinct dimensions
x = torch.randn(2, 4)  # batch_size=2, feature_dim=4
mask = torch.randint(0, 2, (2, 4)).float()  # same shape as x, values 0 or 1

# Define ARGS dict with all parameters
ARGS = {
    "x": x,
    "mask": mask,
    "dim": -1,
    "keepdim": False,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 1, dtype family float, shape (2,) — shape (2,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
neighbor_is_clear Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("d031ea99e1a1dd51")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# Analyze the function:
# - s: needs .shape attribute with [0] and [1] indices (height, width), and indexing with [i, j, k]
# - loc: needs slicing [:2] and indexing [2], appears to be list-like with 3 elements
# - face: integer parameter

# Create a small 3D tensor for s
# Shape: (height=4, width=3, channels=5)
# The function accesses s[neighbor_loc[0], neighbor_loc[1], 4]
# So we need at least 5 channels (indices 0-4)
s = torch.zeros(4, 3, 5)
# Set some values to test the logic
s[1, 1, 4] = 0  # Make this clear (0 means clear)
s[2, 2, 4] = 1  # Make this blocked (1 means blocked)

# Create loc as a list-like object with 3 elements [x, y, orientation]
# x and y should be valid positions in the grid
# orientation (loc[2]) should be 0, 1, 2, or 3 based on the function logic
loc = [1, 1, 0]  # Position (1, 1) with orientation 0

# face parameter - should be 0, 1, or 2 based on the function
face = 0

ARGS = {
    "s": s,
    "loc": loc,
    "face": face
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
state_transition Ran
ran, violated the contract we drafted
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("b7fff67e10e92e79")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 3, dtype family float, shape (5, 6, 16) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# Analyze the function requirements:
# state_transition(s, a, make_error)
# - s: a 3D numpy array with shape (h, w, channels)
#   - First 4 channels store direction state (one-hot for 0-3)
#   - Channel 4 stores obstacle/wall info
#   - Channels 5+ store marker counts (at least MAX_NUM_MARKER + 1 = 11 channels total)
# - a: an action index (0-4)
# - make_error: a boolean flag

# Create a valid state array
h, w = 5, 6  # Grid dimensions
channels = 16  # 4 (direction) + 1 (obstacle) + 11 (markers)
s = np.zeros((h, w, channels), dtype=bool)

# Set the agent location and direction
# Agent starts at position (2, 2) facing direction 0 (up)
agent_x, agent_y = 2, 2
s[agent_x, agent_y, 0] = True  # Direction 0

# Initialize marker count at agent location (0 markers)
s[agent_x, agent_y, 5] = True  # 0 markers (argmax will return 0)

# Add some obstacles to make it interesting
s[1, 2, 4] = True  # Obstacle at (1, 2)

# Valid action: 0 (move), 1 (turn left), 2 (turn right), 3 (pick marker), 4 (put marker)
a = 1

# Flag for error handling
make_error = False

ARGS = {
    "s": s,
    "a": a,
    "make_error": make_error
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ndarray, ndim 3, dtype family float, shape (5, 6, 16) — shape (5, 6, 16) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ConditionPolicy Not yet run
unverified (harvested, no recorded run)
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("9c0f14ab711773c9")
How it ran

no invocation recorded.

Decoder Not yet run
unverified (harvested, no recorded run)
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("692eaa2dd0184a17")
How it ran

no invocation recorded.

ProgramVAE Not yet run
unverified (harvested, no recorded run)
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("49ce8132f623d5b6")
How it ran

no invocation recorded.

VAE Not yet run
unverified (harvested, no recorded run)
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("5f52cac9a31533ed")
How it ran

no invocation recorded.

check_type Not yet run
unverified (harvested, no recorded run)
clvrai/leaps/pretrain/models.py
code served (permissive licence) · get_code("5e9899cd4f3029e3")
How it ran

no invocation recorded.

Also present in other papers' code

Of 20 samples lifted from this paper's linked repositories, 20 carry a function identity and 0 do not. Of the 20 distinct functions with an identity, 0 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 0 later) and 20 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-21.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Recently, deep reinforcement learning (DRL) methods have achieved impressive performance on tasks in a variety of domains. However, neural network policies produced with DRL methods are not human-interpretable and often have difficulty generalizing to novel scenarios. To address these issues, prior works explore learning programmatic policies that are more interpretable and structured for generalization. Yet, these works either employ limited policy representations (e.g. decision trees, state machines, or predefined program templates) or require stronger supervision (e.g. input/output state pairs or expert demonstrations). We present a framework that instead learns to synthesize a program, which details the procedure to solve a task in a flexible and expressive manner, solely from reward signals. To alleviate the difficulty of learning to compose programs to induce the desired agent behavior from scratch, we propose to first learn a program embedding space that continuously parameterizes diverse behaviors in an unsupervised manner and then search over the learned program embedding space to yield a program that maximizes the return for a given task. Experimental results demonstrate that the proposed framework not only learns to reliably synthesize task-solving programs but also outperforms DRL and program synthesis baselines while producing interpretable and more generalizable policies. We also justify the necessity of the proposed two-stage learning scheme as well as analyze various methods for learning the program embedding. Website at https://clvrai.com/leaps. * Contributed equally. † Work partially done as a visiting scholar at USC.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2108.13643")
get_code_for_paper("2108.13643")
have("2108.13643")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2108.13643.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2108.13643.svg)](https://syntology.ai/paper/2108.13643)

Connect an agent — have() is free.