| adjust_groundtruth |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/analyzer/import_datasets.py code served (permissive licence) · get_code("a699801fd94d79f9") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# Analyze the function:
# - Takes 'assignment' parameter
# - Checks that each element x is either None, int, or non-empty list
# - Returns a list where ints are wrapped in lists, and lists/Nones pass through
# Create a valid assignment with mixed types
assignment = [
1, # int - will be wrapped in [1]
None, # None - will pass through as None
[2, 3, 4], # non-empty list - will pass through
5, # int - will be wrapped in [5]
[10], # non-empty list - will pass through
]
ARGS = {
"assignment": assignment
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list — a list output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| cleanup_value |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/analyzer/export.py code served (permissive licence) · get_code("a2e05f3e195e35e1") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import math
def cleanup_value(value):
return value if value is not None else math.inf
ARGS = {
"value": None
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| compute_area |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/perf.py code served (permissive licence) · get_code("f89988646272041a") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# Construct method_data dict with 'x' and 'y' keys
# x should be sorted and all values <= max_perf_ratio
# y should have same length as x
x_values = [0.0, 0.5, 0.8, 1.0]
y_values = [0.2, 0.5, 0.9, 0.95]
method_data = {
'x': x_values,
'y': y_values
}
# max_perf_ratio should be >= max(x) to satisfy the assertion
max_perf_ratio = 1.2
ARGS = {
'method_data': method_data,
'max_perf_ratio': max_perf_ratio
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| compute_area |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/analyzer/plot_perf.py code served (permissive licence) · get_code("9e51883f9e53fa97") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# Construct valid arguments for compute_area
# The function expects method_data dict with 'x' and 'y' keys
# where x and y are sequences of numbers
method_data = {
'x': [0.0, 1.0, 2.0, 3.0],
'y': [1.0, 2.0, 3.0, 4.0]
}
ARGS = {
'method_data': method_data
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| compute_performance_plot_data |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/perf.py code served (permissive licence) · get_code("8afcb50b4993347c") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import sqlite3
from contextlib import contextmanager
# Create an in-memory SQLite database with the required schema and test data
def create_test_db():
db = sqlite3.connect(':memory:')
db.row_factory = sqlite3.Row
# Create tables
db.execute('''
CREATE TABLE instance (
id INTEGER PRIMARY KEY,
dataset_id INTEGER,
optimum REAL
)
''')
db.execute('''
CREATE TABLE dataset (
id INTEGER PRIMARY KEY,
name TEXT
)
''')
db.execute('''
CREATE TABLE method (
id INTEGER PRIMARY KEY,
name TEXT
)
''')
db.execute('''
CREATE TABLE output_postprocessed (
run_id INTEGER,
method_id INTEGER,
instance_id INTEGER,
time REAL,
value REAL
)
''')
# Insert test data
db.execute("INSERT INTO dataset VALUES (1, 'test_dataset')")
db.execute("INSERT INTO instance VALUES (1, 1, 100.0)")
db.execute("INSERT INTO instance VALUES (2, 1, 200.0)")
db.execute("INSERT INTO instance VALUES (3, 1, 150.0)")
db.execute("INSERT INTO method VALUES (1, 'method_a')")
db.execute("INSERT INTO method VALUES (2, 'method_b')")
# Insert output data with varying performance ratios
# Run ID 1, method 1, instance 1
db.execute("INSERT INTO output_postprocessed VALUES (1, 1, 1, 0.015, 95.0)")
db.execute("INSERT INTO output_postprocessed VALUES (1, 1, 2, 0.020, 190.0)")
db.execute("INSERT INTO output_postprocessed VALUES (1, 1, 3, 0.018, 145.0)")
# Run ID 1, method 2, instance 1
db.execute("INSERT INTO output_postprocessed VALUES (1, 2, 1, 0.050, 100.5)")
db.execute("INSERT INTO output_postprocessed VALUES (1, 2, 2, 0.060, 195.0)")
db.execute("INSERT INTO output_postprocessed VALUES (1, 2, 3, 0.055, 152.0)")
db.commit()
return db
db = create_test_db()
ARGS = {
'db': db,
'run_id': 1,
'dataset_id': 1,
'max_perf_ratio': 10.0,
'min_runtime': 0.01,
'optimality_tolerance': 10.0
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| convert_from_json |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/analyzer/export.py code served (permissive licence) · get_code("f8ad7fbd8e8b0d80") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import json
def convert_from_json(value):
if value is not None:
return json.loads(value)
ARGS = {
"value": json.dumps({"key": "example_value", "number": 42})
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| convert_to_bool |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/analyzer/export.py code served (permissive licence) · get_code("8d957e3e9c68e6a6") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# The function convert_to_bool takes a parameter 'v' and:
# - Checks if v is not None
# - Asserts that v == 0 or v == 1
# - Returns bool(v)
# So v should be either 0 or 1 (or None)
ARGS = {
"v": 1
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| fetch_hardware_id |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/analyzer/import_benchmark.py code served (permissive licence) · get_code("b4436e9c7a69bc88") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned int, ndim 0, dtype family int, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import sqlite3
import tempfile
import os
# Create an in-memory SQLite database with a hardware table
db = sqlite3.connect(':memory:')
cursor = db.cursor()
# Create the hardware table with id and name columns
cursor.execute('''
CREATE TABLE hardware (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL
)
''')
# Insert some test data
cursor.execute("INSERT INTO hardware (id, name) VALUES (1, 'GPU')")
cursor.execute("INSERT INTO hardware (id, name) VALUES (2, 'CPU')")
cursor.execute("INSERT INTO hardware (id, name) VALUES (3, 'TPU')")
db.commit()
# Define the arguments dictionary
ARGS = {
'db': db,
'hardware': 'GPU'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type int, ndim 0, dtype family int, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_result |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/analyzer/generate_table.py code served (permissive licence) · get_code("698c78a80ac5ab35") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Result. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import sqlite3
from collections import namedtuple
# Create an in-memory SQLite database
db = sqlite3.connect(':memory:')
# Create the benchmark_per_dataset_pretty table
db.execute('''
CREATE TABLE benchmark_per_dataset_pretty (
run_id INTEGER,
method TEXT,
dataset TEXT,
checkpoint TEXT,
value_avg REAL,
bound_avg REAL,
feasible INTEGER,
total INTEGER,
optimal INTEGER,
optima_known INTEGER,
accuracy_known_nodes_avg REAL
)
''')
# Insert a test row
db.execute('''
INSERT INTO benchmark_per_dataset_pretty
(run_id, method, dataset, checkpoint, value_avg, bound_avg, feasible, total, optimal, optima_known, accuracy_known_nodes_avg)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
''', (1, 'test_method', 'test_dataset', 'checkpoint_001', 42.5, 41.2, 1, 100, 95, 1, 0.95))
db.commit()
# Define ARGS
ARGS = {
'db': db,
'run_id': 1,
'method': 'test_method',
'dataset': 'test_dataset',
'checkpoint': 'checkpoint_001'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Result — a Result output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a Result output the recorded ndim 1 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| init_subparser |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/analyzer/add_hardware.py code served (permissive licence) · get_code("422c465477a639b9") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ArgumentParser. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import argparse
# Create a subparsers object from ArgumentParser
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()
ARGS = {
'subparsers': subparsers
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ArgumentParser — a ArgumentParser output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a ArgumentParser output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| init_subparser |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/analyzer/generate_table.py code served (permissive licence) · get_code("25a24527466b0037") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ArgumentParser. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import argparse
# Create a mock subparsers object
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()
ARGS = {
'subparsers': subparsers
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ArgumentParser — a ArgumentParser output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a ArgumentParser output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| init_subparser |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/analyzer/import_benchmark.py code served (permissive licence) · get_code("dd55277f60012391") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ArgumentParser. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import argparse
class MockSubparsers:
def __init__(self):
self.parsers = {}
def add_parser(self, name):
parser = argparse.ArgumentParser()
self.parsers[name] = parser
return parser
subparsers = MockSubparsers()
ARGS = {
'subparsers': subparsers
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ArgumentParser — a ArgumentParser output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a ArgumentParser output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| init_subparser |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/analyzer/import_datasets.py code served (permissive licence) · get_code("e4882065c0be13f4") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ArgumentParser. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import argparse
class MockSubparsers:
def __init__(self):
self.parser = None
def add_parser(self, name):
self.parser = argparse.ArgumentParser(prog=name)
return self.parser
subparsers = MockSubparsers()
ARGS = {
'subparsers': subparsers
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ArgumentParser — a ArgumentParser output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a ArgumentParser output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| init_subparser |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/analyzer/init_database.py code served (permissive licence) · get_code("f1b899fcdb5fb448") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ArgumentParser. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import argparse
torch_manual_seed = __import__('torch').manual_seed(0)
class MockSubparsers:
def __init__(self):
self.parsers = {}
def add_parser(self, name):
parser = argparse.ArgumentParser()
self.parsers[name] = parser
return parser
ARGS = {
'subparsers': MockSubparsers()
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ArgumentParser — a ArgumentParser output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a ArgumentParser output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| init_subparser |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/analyzer/plot_cactus.py code served (permissive licence) · get_code("dd52eec8897ae8e1") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ArgumentParser. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import argparse
# Create a subparsers object from ArgumentParser
main_parser = argparse.ArgumentParser()
subparsers = main_parser.add_subparsers()
# Define ARGS dict with the required parameter
ARGS = {
'subparsers': subparsers
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ArgumentParser — a ArgumentParser output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a ArgumentParser output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| init_subparser |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/analyzer/plot_perf.py code served (permissive licence) · get_code("c37bbbd3f9f1a1c2") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ArgumentParser. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import argparse
class MockSubparsers:
def add_parser(self, name):
return argparse.ArgumentParser()
subparsers = MockSubparsers()
ARGS = {
'subparsers': subparsers
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ArgumentParser — a ArgumentParser output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a ArgumentParser output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| make_rectangular |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/analyzer/plot_perf.py code served (permissive licence) · get_code("726cad3d8f3c2da3") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# method_data is a dict with 'x' and 'y' keys
# The function expects x and y to be lists (or list-like) with equal length
# It iterates through them with range(len(x)) and accesses elements by index
method_data = {
'x': [1.0, 2.0, 3.0],
'y': [10.0, 20.0, 30.0]
}
ARGS = {
'method_data': method_data
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 2, dtype family float — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| missing_instance_in_run |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/verification.py code served (permissive licence) · get_code("848358a4327596b7") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import sqlite3
import tempfile
import os
# Create an in-memory SQLite database
db = sqlite3.connect(':memory:')
# Create the verification_missing_instance_in_run table
db.execute('''
CREATE TABLE verification_missing_instance_in_run (
id INTEGER PRIMARY KEY,
run_id INTEGER,
instance_id INTEGER,
data TEXT
)
''')
# Insert some test data
db.execute('''
INSERT INTO verification_missing_instance_in_run (run_id, instance_id, data)
VALUES (1, 100, 'test_data_1')
''')
db.execute('''
INSERT INTO verification_missing_instance_in_run (run_id, instance_id, data)
VALUES (1, 101, 'test_data_2')
''')
db.commit()
ARGS = {
'db': db,
'run_id': 1
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| missing_method_in_run |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/verification.py code served (permissive licence) · get_code("9c7771a3a2e3c00f") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import sqlite3
import tempfile
import os
# Create an in-memory SQLite database
db = sqlite3.connect(':memory:')
# Create the table that the function expects
db.execute('''
CREATE TABLE verification_missing_method_in_run (
id INTEGER PRIMARY KEY,
run_id INTEGER,
data TEXT
)
''')
# Insert some test data
db.execute('INSERT INTO verification_missing_method_in_run (run_id, data) VALUES (1, "test_data_1")')
db.execute('INSERT INTO verification_missing_method_in_run (run_id, data) VALUES (1, "test_data_2")')
db.commit()
ARGS = {
'db': db,
'run_id': 1
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| missing_trial_in_run |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/verification.py code served (permissive licence) · get_code("2842a1369540b274") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import sqlite3
import tempfile
import os
# Create a temporary in-memory SQLite database
db = sqlite3.connect(':memory:')
# Create the verification_missing_trial_in_run table
db.execute('''
CREATE TABLE verification_missing_trial_in_run (
id INTEGER PRIMARY KEY,
run_id INTEGER,
trial_id INTEGER,
data TEXT
)
''')
# Insert some test data
db.execute('INSERT INTO verification_missing_trial_in_run (run_id, trial_id, data) VALUES (1, 100, "test1")')
db.execute('INSERT INTO verification_missing_trial_in_run (run_id, trial_id, data) VALUES (1, 101, "test2")')
db.execute('INSERT INTO verification_missing_trial_in_run (run_id, trial_id, data) VALUES (2, 200, "test3")')
db.commit()
ARGS = {
'db': db,
'run_id': 1
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| natural_sort |
Ran ran; our contract draft was wrong, not the code |
vislearn/gmbench/python/gmbench/analyzer/import_datasets.py code served (permissive licence) · get_code("1677ad306233c7b7") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import re
def natural_sort(l):
convert = lambda text: int(text) if text.isdigit() else text.lower()
alphanum_key = lambda key: [convert(c) for c in re.split('([0-9]+)', key)]
return sorted(l, key=alphanum_key)
# The function takes a list `l` and sorts it using natural/alphanumeric sorting
# It expects strings as elements in the list
ARGS = {
"l": ["file10.txt", "file2.txt", "file1.txt", "file20.txt", "item3", "item1", "item20"]
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| parse_data_file |
Ran ran on a synthesized input |
vislearn/gmbench/python/gmbench/analyzer/import_benchmark.py code served (permissive licence) · get_code("ea9cacb97f9a2f4b") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import gzip
import json
import tempfile
import os
# Create a temporary gzip file with valid JSON data
temp_dir = tempfile.mkdtemp()
temp_file = os.path.join(temp_dir, "test_data.json.gz")
# Write test data to the gzip file
test_data = {
"key1": "value1",
"key2": [1, 2, 3],
"key3": {"nested": "object"}
}
with gzip.open(temp_file, 'wt') as f:
json.dump(test_data, f)
ARGS = {
"filename": temp_file
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|