| byte_len |
Ran ran on a synthesized input |
ffaisal93/sd-qa/baselines/tydiqa/baseline/data.py code served (permissive licence) · get_code("7671754d0cee7d11") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned int, ndim 0, dtype family int, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
def byte_str(text):
return text.encode('utf-8')
def byte_len(text):
return len(byte_str(text))
# The function byte_len takes a single parameter 'text' which should be a string
# It encodes the string to UTF-8 bytes and returns the length
ARGS = {
'text': 'Hello, world!'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type int, ndim 0, dtype family int, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| byte_slice |
Ran ran on a synthesized input |
ffaisal93/sd-qa/baselines/tydiqa/tydi_eval.py code served (permissive licence) · get_code("787fbf63d8c1ab07") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
# Analyze the function:
# - text: a string that will be encoded to UTF-8 bytes
# - start: integer index for byte slicing
# - end: integer index for byte slicing
# The function encodes text to UTF-8 bytes, slices from start to end,
# and attempts to decode back to string (or returns the bytes repr if decode fails)
ARGS = {
"text": "Hello, 世界!",
"start": 0,
"end": 5
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| byte_slice |
Ran ran on a synthesized input |
ffaisal93/sd-qa/baselines/tydiqa/baseline/data.py code served (permissive licence) · get_code("5e27cb5b15ab895f") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# Analyzing the function byte_slice:
# - text: a string that will be encoded to utf-8 bytes, sliced, then decoded
# - start: integer index for slice start
# - end: integer index for slice end
# - errors: string parameter for decode error handling (default 'replace')
ARGS = {
'text': 'Hello, World! 🌍',
'start': 0,
'end': 13,
'errors': 'replace'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| byte_str |
Ran ran on a synthesized input |
ffaisal93/sd-qa/baselines/tydiqa/baseline/data.py code served (permissive licence) · get_code("a5845e18acb8844b") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bytes. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
ARGS = {
"text": "Hello, world!"
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type bytes — a bytes output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a bytes output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| dir_check |
Ran ran on a synthesized input |
ffaisal93/sd-qa/experiments/utils.py code served (permissive licence) · get_code("ea2aa48fb12d0a30") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import os
import tempfile
# Create a temporary directory for testing
with tempfile.TemporaryDirectory() as tmpdir:
# Create a test directory
test_dir = os.path.join(tmpdir, "test_directory")
os.makedirs(test_dir, exist_ok=True)
# Create a test file
test_file = os.path.join(tmpdir, "test_file.txt")
with open(test_file, 'w') as f:
f.write("test content")
# Define arguments for both test cases
ARGS = {
'path': test_dir,
'types': 'dir'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| extract_answers |
Ran ran on a synthesized input |
ffaisal93/sd-qa/extract_qa_from_jsonl.py code served (permissive licence) · get_code("fd24ba8d7375d425") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
ARGS = {
'example': {
'document_plaintext': 'The quick brown fox jumps over the lazy dog. Python is great.',
'annotations': [
{
'minimal_answer': {
'plaintext_start_byte': 4,
'plaintext_end_byte': 9
},
'passage_answer': {
'candidate_index': -1
}
},
{
'minimal_answer': {
'plaintext_start_byte': -1,
'plaintext_end_byte': -1
},
'passage_answer': {
'candidate_index': 0
}
},
{
'minimal_answer': None,
'passage_answer': {
'candidate_index': 1
}
}
],
'passage_answer_candidates': [
{
'plaintext_start_byte': 16,
'plaintext_end_byte': 19
},
{
'plaintext_start_byte': 48,
'plaintext_end_byte': 54
}
]
}
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| gelu |
Ran ran; our contract draft was wrong, not the code |
ffaisal93/sd-qa/baselines/tydiqa/baseline/bert/modeling.py code served (permissive licence) · get_code("51d3bc0cfa6415a3") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned SymbolicTensor. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()
# Create a small float32 tensor for x
x = tf.constant([[0.5, -0.3, 1.2, -0.8],
[0.1, 0.2, -0.5, 0.9]], dtype=tf.float32)
ARGS = {
"x": x
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, tensorflow — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type SymbolicTensor — a SymbolicTensor output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_activation |
Ran ran; our contract draft was wrong, not the code |
ffaisal93/sd-qa/baselines/tydiqa/baseline/bert/modeling.py code served (permissive licence) · get_code("217fe96a5945a42d") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned function. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
import six
import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()
ARGS = {
'activation_string': 'relu'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, six, tensorflow — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type function — a function output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a function output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_assignment_map_from_checkpoint |
Ran ran; our contract draft was wrong, not the code |
ffaisal93/sd-qa/baselines/tydiqa/baseline/bert/modeling.py code served (permissive licence) · get_code("7bc46f4950599a87") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import collections
import re
import tensorflow.compat.v1 as tf
import tempfile
import os
# Disable eager execution for TensorFlow v1 compatibility
tf.disable_eager_execution()
# Create a temporary checkpoint file for testing
temp_dir = tempfile.mkdtemp()
checkpoint_path = os.path.join(temp_dir, "test_checkpoint")
# Create a simple TensorFlow checkpoint with some variables
with tf.Graph().as_default():
with tf.Session() as sess:
# Create some variables to save
var1 = tf.Variable(tf.ones([2, 3]), name="layer1/weights")
var2 = tf.Variable(tf.zeros([4]), name="layer1/bias")
var3 = tf.Variable(tf.ones([3, 4]), name="layer2/weights")
sess.run(tf.global_variables_initializer())
# Save the checkpoint
saver = tf.train.Saver()
saver.save(sess, checkpoint_path)
# Create variables for the current graph
with tf.Graph().as_default():
with tf.Session() as sess:
# Create variables with matching names
tvars = [
tf.Variable(tf.ones([2, 3]), name="layer1/weights"),
tf.Variable(tf.zeros([4]), name="layer1/bias"),
tf.Variable(tf.ones([3, 4]), name="layer2/weights"),
tf.Variable(tf.ones([5]), name="layer3/bias"), # This one won't be in checkpoint
]
sess.run(tf.global_variables_initializer())
ARGS = {
"tvars": tvars,
"init_checkpoint": checkpoint_path,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- tensorflow — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_metadata |
Ran ran on a synthesized input |
ffaisal93/sd-qa/experiments/utils.py code served (permissive licence) · get_code("d60c3869c082ce69") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import os
import pandas as pd
import tempfile
import shutil
# Create a temporary directory structure for testing
temp_dir = tempfile.mkdtemp()
# Create the directory structure: root_path/f1/language/subdirs
f1_name = "test_split"
f1_path = os.path.join(temp_dir, f1_name)
os.makedirs(f1_path)
# Create language directories and CSV files
languages = ['ben', 'eng', 'ara', 'kor', 'swa']
lang_full_names = {'ben': 'bengali', 'eng': 'english', 'ara': 'arabic', 'kor': 'korean', 'swa': 'swahili'}
for lang in languages:
lang_path = os.path.join(f1_path, lang)
os.makedirs(lang_path)
# Create the language CSV file
csv_filename = os.path.join(lang_path, f"{lang_full_names[lang]}.{f1_name}.csv")
# Create sample data
data = {
'Unnamed: 0': [0, 1],
f"{lang_full_names[lang]}-gold": ['word1 ', 'word2'],
}
# Add kenya-en-KE column for eng
if lang == 'eng':
data['kenya-en-KE'] = ['value1', 'value2']
else:
data['kenya-en-KE'] = ['#NOTFOUND#', '#NOTFOUND#']
df = pd.DataFrame(data)
df.to_csv(csv_filename)
# Create subdirectories with metadata files
for subdir_idx in range(2):
subdir_name = f"subset_{subdir_idx}"
subdir_path = os.path.join(lang_path, subdir_name)
os.makedirs(subdir_path)
# Create metadata CSV file
meta_filename = os.path.join(subdir_path, "metadata.csv")
meta_data = {
'Unnamed: 0': [0, 1],
'language': ['swh' if lang == 'swa' else lang, 'swh' if lang == 'swa' else lang],
'other_field': ['val1', 'val2']
}
meta_df = pd.DataFrame(meta_data)
meta_df.to_csv(meta_filename)
# Define discard set
discard = set()
ARGS = {
'root_path': temp_dir,
'f1': f1_name,
'discard': discard
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- pandas — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| load_jsonl_example_ids |
Ran ran on a synthesized input |
ffaisal93/sd-qa/analyze_data_links.py code served (permissive licence) · get_code("da7b79cb32612610") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import json
import tempfile
from pathlib import Path
# Create a temporary directory for test files
temp_dir = Path(tempfile.gettempdir())
# Create a test JSONL file with example_ids
test_jsonl_file = temp_dir / "test_examples.jsonl"
with open(test_jsonl_file, 'w', encoding='utf-8') as f:
f.write(json.dumps({'example_id': 'ex_001', 'data': 'sample'}) + '\n')
f.write(json.dumps({'example_id': 'ex_002', 'data': 'sample2'}) + '\n')
f.write(json.dumps({'example_id': 123, 'data': 'sample3'}) + '\n')
f.write('\n') # Empty line to test filtering
f.write(json.dumps({'data': 'no_id'}) + '\n')
ARGS = {
'jsonl_file': test_jsonl_file,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| load_metadata_ids |
Ran ran on a synthesized input |
ffaisal93/sd-qa/analyze_data_links.py code served (permissive licence) · get_code("50ed7d3089187d7d") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned set. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import csv
from pathlib import Path
import tempfile
# Create a temporary directory and metadata file for testing
temp_dir = tempfile.TemporaryDirectory()
temp_path = Path(temp_dir.name)
metadata_file = temp_path / "metadata.csv"
# Write a sample metadata.csv file with some example_ids
with open(metadata_file, 'w', newline='', encoding='utf-8') as f:
writer = csv.DictWriter(f, fieldnames=['example_id', 'other_field'])
writer.writeheader()
writer.writerow({'example_id': 'id_001', 'other_field': 'value1'})
writer.writerow({'example_id': 'id_002', 'other_field': 'value2'})
writer.writerow({'example_id': ' id_003 ', 'other_field': 'value3'})
writer.writerow({'example_id': '', 'other_field': 'value4'})
ARGS = {
'metadata_file': metadata_file
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type set — a set output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a set output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| load_txt_ids |
Ran ran on a synthesized input |
ffaisal93/sd-qa/analyze_data_links.py code served (permissive licence) · get_code("5d1dc7f1ffae5ff9") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned set. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
from pathlib import Path
import tempfile
import os
# Create a temporary txt file with tab-separated data
temp_dir = tempfile.mkdtemp()
txt_file_path = Path(temp_dir) / "test_ids.txt"
# Write test data to the file
with open(txt_file_path, 'w', encoding='utf-8') as f:
f.write("id1\tvalue1\tvalue2\n")
f.write("id2\tvalue3\tvalue4\n")
f.write("id3\tvalue5\tvalue6\n")
ARGS = {
"txt_file": txt_file_path
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type set — a set output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a set output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| read_reference |
Ran ran on a synthesized input |
ffaisal93/sd-qa/experiments/transcript_replace.py code served (permissive licence) · get_code("91c7ebc9939cc186") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import tempfile
import csv
import os
# Create a temporary CSV file with test data
temp_file = tempfile.NamedTemporaryFile(mode='w', delete=False, suffix='.csv', newline='')
temp_path = temp_file.name
try:
writer = csv.writer(temp_file)
writer.writerow(['ID', 'question', 'other_col'])
writer.writerow(['1', 'What is this', 'data'])
writer.writerow(['2', 'How are you', 'more'])
writer.writerow(['3', 'Is this working', 'stuff'])
temp_file.close()
ARGS = {
'f': temp_path,
'col': 'question',
'qmark': None
}
finally:
pass
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| safe_average |
Ran ran on a synthesized input |
ffaisal93/sd-qa/baselines/tydiqa/eval_utils.py code served (permissive licence) · get_code("e5609b0f6acf5d82") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# safe_average takes a single parameter: elements
# It calls sum(elements) and len(elements), so elements must be an iterable
# Let's create a list of numbers
elements = [1.0, 2.0, 3.0, 4.0, 5.0]
ARGS = {
"elements": elements
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| safe_divide |
Ran ran on a synthesized input |
ffaisal93/sd-qa/baselines/tydiqa/eval_utils.py code served (permissive licence) · get_code("999c2750713a6692") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
ARGS = {
"x": 10,
"y": 2,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| sanitize_field |
Ran ran on a synthesized input |
ffaisal93/sd-qa/extract_qa_from_jsonl.py code served (permissive licence) · get_code("606f24554fa9223b") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
ARGS = {
"s": "hello\tworld\nfoo\r\nbar baz"
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| utf8_slice |
Ran ran on a synthesized input |
ffaisal93/sd-qa/extract_qa_from_jsonl.py code served (permissive licence) · get_code("812e406d9764d77d") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
# Analyze the function utf8_slice:
# - text: str - a string to slice
# - start: int - byte offset for start (inclusive)
# - end: int - byte offset for end (exclusive)
# The function slices UTF-8 encoded bytes and decodes back to string
# Create test arguments
text_value = "Hello, 世界! This is UTF-8 text."
start_value = 0
end_value = 13
ARGS = {
"text": text_value,
"start": start_value,
"end": end_value,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| utf8_slice |
Ran ran on a synthesized input |
ffaisal93/sd-qa/sdqa_triples_from_jsonl.py code served (permissive licence) · get_code("f4a930be2113e081") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
# Construct arguments for utf8_slice function
# The function takes: text (str), start (int), end (int)
text = "Hello, 世界! 🌍" # Mixed ASCII and Unicode characters
start = 0
end = 20
ARGS = {
"text": text,
"start": start,
"end": end
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| write_out |
Ran ran on a synthesized input |
ffaisal93/sd-qa/experiments/transcript_replace.py code served (permissive licence) · get_code("dad8041c5ab85e66") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import json
import torch
import numpy as np
torch.manual_seed(0)
# The function write_out takes a single parameter 'line' and converts it to JSON
# It uses json.dumps with ensure_ascii=False
# So 'line' should be a JSON-serializable object
# Test with various types that json.dumps can handle
test_line = {
"text": "Hello, world!",
"number": 42,
"float": 3.14,
"list": [1, 2, 3],
"nested": {"key": "value"},
"unicode": "こんにちは"
}
ARGS = {
"line": test_line
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| compute_partial_match_scores |
Not yet run unverified (harvested, no recorded run) |
ffaisal93/sd-qa/baselines/tydiqa/eval_utils.py code served (permissive licence) · get_code("7ef49e1793449f8b") How it ran |
| find_audio_dir |
Not yet run unverified (harvested, no recorded run) |
ffaisal93/sd-qa/sdqa_triples_from_jsonl.py code served (permissive licence) · get_code("d19afd0251c48ecd") How it ran |