| action_dist |
Ran ran, honoured the contract we drafted |
zharry29/nl-to-pddl/source/pddl-parser/pddl_parser/action.py code served (permissive licence) · get_code("5b6ff36d5b5bdb27") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
class Action:
def __init__(self, parameters, positive_preconditions, negative_preconditions, add_effects, del_effects):
self.parameters = parameters
self.positive_preconditions = positive_preconditions
self.negative_preconditions = negative_preconditions
self.add_effects = add_effects
self.del_effects = del_effects
# Create two Action instances with matching structure
Action1 = Action(
parameters=[("param_a", "type_1"), ("param_b", "type_2")],
positive_preconditions=[("predicate_1", "param_a"), ("predicate_2", "param_b")],
negative_preconditions=[("predicate_3", "param_a")],
add_effects=[("effect_1", "param_a"), ("effect_2", "param_b")],
del_effects=[("effect_3", "param_a")]
)
Action2 = Action(
parameters=[("param_x", "type_1"), ("param_y", "type_2"), ("param_z", "type_3")],
positive_preconditions=[("predicate_1", "param_x"), ("predicate_4", "param_y")],
negative_preconditions=[("predicate_5", "param_x")],
add_effects=[("effect_1", "param_x")],
del_effects=[("effect_4", "param_y")]
)
ARGS = {
"Action1": Action1,
"Action2": Action2
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family int — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| action_equal |
Ran ran, violated the contract we drafted |
zharry29/nl-to-pddl/source/pddl-parser/pddl_parser/action.py code served (permissive licence) · get_code("34f579fa6aeca2a2") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
class Action:
def __init__(self, parameters, positive_preconditions, negative_preconditions, add_effects, del_effects):
self.parameters = parameters
self.positive_preconditions = positive_preconditions
self.negative_preconditions = negative_preconditions
self.add_effects = add_effects
self.del_effects = del_effects
# Create two identical Action objects for a successful comparison
parameters1 = [("x", "location"), ("y", "object")]
parameters2 = [("x", "location"), ("y", "object")]
positive_preconditions1 = [("at", "x"), ("holding", "y")]
positive_preconditions2 = [("at", "x"), ("holding", "y")]
negative_preconditions1 = [("moving", "x")]
negative_preconditions2 = [("moving", "x")]
add_effects1 = [("carried", "y")]
add_effects2 = [("carried", "y")]
del_effects1 = [("free", "y")]
del_effects2 = [("free", "y")]
Action1 = Action(
parameters=parameters1,
positive_preconditions=positive_preconditions1,
negative_preconditions=negative_preconditions1,
add_effects=add_effects1,
del_effects=del_effects1
)
Action2 = Action(
parameters=parameters2,
positive_preconditions=positive_preconditions2,
negative_preconditions=negative_preconditions2,
add_effects=add_effects2,
del_effects=del_effects2
)
ARGS = {
"Action1": Action1,
"Action2": Action2
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| action_instance_comp |
Ran ran, violated the contract we drafted |
zharry29/nl-to-pddl/source/pddl_evaluation/src/compare_plan.py code served (permissive licence) · get_code("a339438fceb905a1") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import collections
def setup_action_instances():
# action_instance format: (element1, list_or_iterable)
# The function compares:
# 1. First elements for equality
# 2. Counters of second elements for equality
action_instance1 = ("action_type_A", ["item1", "item2", "item1"])
action_instance2 = ("action_type_A", ["item1", "item1", "item2"])
return action_instance1, action_instance2
action_instance1, action_instance2 = setup_action_instances()
ARGS = {
"action_instance1": action_instance1,
"action_instance2": action_instance2,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| captialize_each_word |
Ran ran; our contract draft was wrong, not the code |
zharry29/nl-to-pddl/source/pddl_evaluation/src/util.py code served (permissive licence) · get_code("c7d2ee1f8f892828") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
ARGS = {
'input': 'hello world python code'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| compare_actions_f1 |
Ran ran, honoured the contract we drafted |
zharry29/nl-to-pddl/source/pddl_evaluation/src/compare_plan.py code served (permissive licence) · get_code("0589936b54490eaa") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import collections
def parse_action(action_str):
action_name = action_str.split(" ('")[0]
arguments = action_str.split(" ('")[1].split("')")[0].split("', '")
return (action_name, arguments)
def action_instance_comp(action_instance1, action_instance2):
return action_instance1[0] == action_instance2[0] and collections.Counter(action_instance1[1]) == collections.Counter(action_instance2[1])
def compare_actions_f1(pred_actions, gold_actions):
pred_actions = [parse_action(action) for action in pred_actions if action != '']
gold_actions = [parse_action(action) for action in gold_actions if action != '']
correct_pred, correct_gold = (0, 0)
all_pred, all_gold = (len(pred_actions), len(gold_actions))
for pred_action in pred_actions:
for gold_action in gold_actions:
if action_instance_comp(pred_action, gold_action):
correct_pred += 1
break
for gold_action in gold_actions:
for pred_action in pred_actions:
if action_instance_comp(pred_action, gold_action):
correct_gold += 1
break
precision = correct_pred / all_pred
recall = correct_gold / all_gold
f1 = 2 * precision * recall / (precision + recall)
return (precision, recall, f1)
# Setup test data
# Action strings follow format: "action_name ('arg1', 'arg2', ...)"
pred_actions = [
"click ('button1', 'submit')",
"type ('field1', 'hello')",
"navigate ('page2')"
]
gold_actions = [
"click ('button1', 'submit')",
"type ('field1', 'hello')",
"scroll ('down')"
]
ARGS = {
"pred_actions": pred_actions,
"gold_actions": gold_actions
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_action_names |
Ran ran; our contract draft was wrong, not the code |
zharry29/nl-to-pddl/source/pddl_evaluation/src/make_gold.py code served (permissive licence) · get_code("011b75c04a7466f1") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
# Analyze the function: get_action_names(domain)
# The function takes a 'domain' parameter and:
# 1. Splits it by '\n'
# 2. Strips whitespace from each line
# 3. Filters lines that start with '(:action'
# 4. Joins them with '\n'
# So 'domain' must be a string containing PDDL domain definition text
domain_text = """(define (domain logistics)
(:requirements :typing :action-costs)
(:types truck location obj)
(:action load
:parameters (?o - obj ?t - truck ?l - location)
:precondition (and (at ?o ?l) (at ?t ?l))
:effect (and (not (at ?o ?l)) (in ?o ?t)))
(:action unload
:parameters (?o - obj ?t - truck ?l - location)
:precondition (and (in ?o ?t) (at ?t ?l))
:effect (and (at ?o ?l) (not (in ?o ?t))))
(:action drive
:parameters (?t - truck ?from ?to - location)
:precondition (at ?t ?from)
:effect (and (not (at ?t ?from)) (at ?t ?to)))
)
"""
ARGS = {
"domain": domain_text
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_actions |
Ran ran; our contract draft was wrong, not the code |
zharry29/nl-to-pddl/source/pddl_evaluation/src/make_gold.py code served (permissive licence) · get_code("4bf89936621523df") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
# The function takes a 'domain' parameter which is a string
# Looking at the function body, it:
# 1. Splits the string by '(:action'
# 2. Joins parts back with '(:action'
# 3. Splits by '\n' and processes lines
# 4. Checks if lines start with ' ' (3 spaces)
# 5. Returns a reformatted string
# Create a sample domain string with action definitions
domain = """(:domain test-domain
(:requirements :strips :typing)
(:types location robot)
(:action move
:parameters (?r - robot ?from - location ?to - location)
:precondition (at ?r ?from)
:effect (and (at ?r ?to) (not (at ?r ?from))))
(:action pick
:parameters (?r - robot ?obj - object)
:precondition (at ?r ?obj)
:effect (holding ?r ?obj)))"""
ARGS = {
'domain': domain
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_domain_header |
Ran ran; our contract draft was wrong, not the code |
zharry29/nl-to-pddl/source/pddl_evaluation/src/make_gold.py code served (permissive licence) · get_code("a23888f4b12c48f9") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
def get_domain_header(domain):
return domain.split('(:action')[0].strip()
# The function takes a single parameter 'domain' which is a string
# It splits on '(:action' and returns the first part, stripped of whitespace
domain_string = "domain blocksworld (:action pick-up"
ARGS = {
"domain": domain_string
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| load_json |
Ran ran; our contract draft was wrong, not the code |
zharry29/nl-to-pddl/source/pddl_evaluation/src/util.py code served (permissive licence) · get_code("550dd4e385e290a6") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import json
import tempfile
import os
# Create a temporary JSON file with test data
temp_dir = tempfile.mkdtemp()
test_json_file = os.path.join(temp_dir, "test.json")
test_data = {"key": "value", "number": 42, "list": [1, 2, 3]}
with open(test_json_file, 'w') as f:
json.dump(test_data, f)
# Define ARGS with the filename parameter
ARGS = {
"filename": test_json_file
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| load_txt_prompt |
Ran ran; our contract draft was wrong, not the code |
zharry29/nl-to-pddl/source/pddl_evaluation/src/util.py code served (permissive licence) · get_code("d136846408593074") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import tempfile
import os
# Create a temporary text file with sample content
temp_dir = tempfile.mkdtemp()
temp_filename = os.path.join(temp_dir, "prompt.txt")
with open(temp_filename, 'w') as f:
f.write("This is a sample prompt.\n")
f.write("It contains multiple lines.\n")
f.write("And various content.")
ARGS = {
"filename": temp_filename
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| parse_action |
Ran ran; our contract draft was wrong, not the code |
zharry29/nl-to-pddl/source/pddl_evaluation/src/compare_plan.py code served (permissive licence) · get_code("9fc0305b3d6abb02") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
def parse_action(action_str):
action_name = action_str.split(" ('")[0]
arguments = action_str.split(" ('")[1].split("')")[0].split("', '")
return (action_name, arguments)
# The function takes a single parameter: action_str
# It expects a string formatted as: "action_name ('arg1', 'arg2', ...)"
# The function parses this string to extract the action name and arguments list
action_str = "move ('left', 'forward')"
ARGS = {
"action_str": action_str
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple — a tuple output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| action_dist |
Not yet run unverified (harvested, no recorded run) |
zharry29/nl-to-pddl/source/pddl-parser/pddl_parser/action.py code served (permissive licence) · get_code("ddb36e302e1fe2dc") How it ran |