| compute_amm_and_diversity |
Ran ran, honoured the contract we drafted |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("eea19ee437148fef") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import cv2
import numpy as np
import tempfile
import os
# Create temporary image files for testing
temp_dir = tempfile.mkdtemp()
# Create 3 simple test images (small RGB images)
image_paths = []
for i in range(3):
# Create a simple 100x100 BGR image
img = np.random.randint(0, 256, (100, 100, 3), dtype=np.uint8)
img_path = os.path.join(temp_dir, f"frame_{i}.png")
cv2.imwrite(img_path, img)
image_paths.append(img_path)
ARGS = {
"paths": image_paths,
"max_frames": 2
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- cv2, numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| is_oom_error |
Ran ran, violated the contract we drafted |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("2cafe521fc07a22c") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# The function takes an Exception and checks if it's an out-of-memory error
# We need to construct various Exception instances to test the function
# Test case 1: Exception with 'out of memory' in the message
exc_with_oom_msg = Exception("CUDA out of memory error")
# Test case 2: Regular exception without OOM message
exc_regular = Exception("Some other error")
# Test case 3: Exception with lowercase 'out of memory'
exc_lowercase_oom = RuntimeError("out of memory on device")
# We'll use the first one as the main test case
ARGS = {
"exc": exc_with_oom_msg
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| iter_samples |
Ran ran; our contract draft was wrong, not the code |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("420619a45dd0436c") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned generator. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import tempfile
import json
import os
# Create temporary files for onevision and video data
temp_dir = tempfile.mkdtemp()
onevision_path = os.path.join(temp_dir, 'onevision.jsonl')
video_path = os.path.join(temp_dir, 'video.jsonl')
# Write sample onevision data
onevision_data = [
{
'id': 'ov_001',
'images': ['image1.jpg', 'image2.jpg'],
'messages': [
{'role': 'user', 'content': 'What is in this image?'},
{'role': 'assistant', 'content': 'This is a test image.'},
{'role': 'human', 'content': 'Tell me more <image>'},
{'role': 'gpt', 'content': 'There is more detail here.'}
]
},
{
'id': 'ov_002',
'images': ['image3.jpg'],
'messages': [
{'role': 'from', 'value': 'Describe the scene'},
{'role': 'gpt', 'value': 'A scenic landscape'}
]
}
]
with open(onevision_path, 'w', encoding='utf-8') as f:
for item in onevision_data:
f.write(json.dumps(item) + '\n')
# Write sample video data
video_data = [
{
'id': 'vid_001',
'video_id': 'video_uid_001',
'images': ['frame1.jpg', 'frame2.jpg', 'frame3.jpg', 'frame4.jpg', 'frame5.jpg'],
'messages': [
{'role': 'user', 'content': 'What happens in this video? <video>'},
{'role': 'assistant', 'content': 'The video shows action.'},
{'role': 'human', 'content': 'Any other details?'},
{'role': 'gpt', 'content': 'Yes, there are many details.'}
]
},
{
'id': 'vid_002',
'video_id': 'video_uid_002',
'images': ['frame6.jpg', 'frame7.jpg'],
'messages': [
{'role': 'from', 'value': 'Analyze the video <video>'},
{'role': 'gpt', 'value': 'The analysis is complete.'}
]
}
]
with open(video_path, 'w', encoding='utf-8') as f:
for item in video_data:
f.write(json.dumps(item) + '\n')
# Define ARGS dictionary for iter_samples function
ARGS = {
'onevision_path': onevision_path,
'video_path': video_path,
'target_frames': 3,
'onevision_max_lines': 10,
'video_max_lines': 10,
'onevision_total_lines': 2,
'video_total_lines': 2,
'log_every': 1,
'enable_tqdm': False,
'print_parsing_log': False
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type generator — a generator output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a generator output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| jaccard_similarity |
Ran ran, honoured the contract we drafted |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("cd6a7e9e2afeead9") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import re
from typing import Dict, Iterator, List, Optional, Sequence, Tuple
WORD_RE = re.compile('[A-Za-z0-9_]+')
def tokenize(text: str) -> List[str]:
return WORD_RE.findall((text or '').lower())
def jaccard_similarity(a: str, b: str) -> float:
ta = set(tokenize(a))
tb = set(tokenize(b))
if not ta and (not tb):
return 1.0
inter = len(ta & tb)
union = len(ta | tb)
return float(inter / max(union, 1))
ARGS = {
"a": "hello world test",
"b": "hello python world"
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| load_pil_images |
Ran ran, honoured the contract we drafted |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("08ef9725aaf447ef") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned list, ndim 4, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import tempfile
import os
from PIL import Image
import numpy as np
# Create temporary directory with test images
temp_dir = tempfile.mkdtemp()
# Create a few test images
test_image_paths = []
for i in range(3):
# Create a simple RGB image using PIL
img_array = np.random.randint(0, 256, (10, 10, 3), dtype=np.uint8)
img = Image.fromarray(img_array, mode='RGB')
img_path = os.path.join(temp_dir, f'test_image_{i}.png')
img.save(img_path)
test_image_paths.append(img_path)
# Add a non-existent path to test exception handling
test_image_paths.append(os.path.join(temp_dir, 'nonexistent.png'))
ARGS = {
'paths': test_image_paths
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- PIL, numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 4, dtype family int — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| make_texts |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("3e2ba7ffc4752aa5") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from transformers import AutoProcessor
from unittest.mock import Mock, MagicMock
torch.manual_seed(0)
# Create a minimal mock processor that implements apply_chat_template
class MockProcessor:
def apply_chat_template(self, messages, tokenize=False, add_generation_prompt=False):
# Simple mock implementation that returns a string representation
if add_generation_prompt:
return "<|im_start|>user\nquestion<|im_end|>\n<|im_start|>assistant\n"
else:
return "<|im_start|>user\nquestion<|im_end|>\n<|im_start|>assistant\nanswer<|im_end|>"
processor = MockProcessor()
ARGS = {
'processor': processor,
'question': 'What is in this image?',
'answer': 'This is a test answer.',
'num_images': 2,
'use_images': True,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| normalize_text |
Ran ran; our contract draft was wrong, not the code |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("db008612f143341f") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# The function normalize_text takes a single parameter 'x' of type object
# It handles None, converts to string, strips whitespace, and joins with single spaces
# Test with various input types to ensure robustness
ARGS = {
'x': ' hello world \n test '
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| parse_message_pairs |
Ran ran; our contract draft was wrong, not the code |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("7b202711adf7f2cd") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned list, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
from typing import Dict, List, Tuple
torch.manual_seed(0)
# Construct valid arguments for parse_message_pairs
ARGS = {
'messages': [
{
'role': 'user',
'content': 'Hello, what is <image> about?'
},
{
'role': 'assistant',
'content': 'This is a response about the image.'
},
{
'role': 'human',
'content': 'Tell me more about <video> content.'
},
{
'role': 'gpt',
'content': 'The video shows interesting things.'
},
{
'role': 'user',
'content': 'What about <audio>?'
},
{
'role': 'assistant',
'content': 'The audio contains speech and music.'
}
]
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 2, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| read_frames |
Ran ran, honoured the contract we drafted |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("edaed27bb877f0f0") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 4, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
import tempfile
import os
# Create temporary image files for testing
temp_dir = tempfile.mkdtemp()
# Create a few dummy image files using numpy (simulating cv2.imread output)
# We'll create simple dummy files that cv2.imread can attempt to read
dummy_images = []
for i in range(5):
# Create a simple 10x10 RGB image as numpy array
img_array = np.random.randint(0, 255, (10, 10, 3), dtype=np.uint8)
img_path = os.path.join(temp_dir, f"image_{i}.png")
# Write using cv2 if available, otherwise just store paths
try:
import cv2
cv2.imwrite(img_path, img_array)
dummy_images.append(img_path)
except Exception:
dummy_images.append(img_path)
# Construct ARGS dict with valid arguments
ARGS = {
"paths": dummy_images, # Sequence of valid image file paths
"max_frames": 3 # Target number of frames to sample
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- cv2, numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 4, dtype family int — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| sample_paths |
Ran ran; our contract draft was wrong, not the code |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("c4fb4107dc8b9208") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# Analyze the function:
# - paths: Sequence[str] - a sequence of strings
# - target: int - an integer target count
# The function filters out falsy values, converts to strings, and samples indices
ARGS = {
"paths": ["path1", "path2", "path3", "path4", "path5"],
"target": 3
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| self_consistency_score |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("e5992b29a0260e38") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from PIL import Image
from transformers import AutoProcessor
from unittest.mock import MagicMock, patch
from dataclasses import dataclass
torch.manual_seed(0)
@dataclass
class ModelBundle:
processor: AutoProcessor
model: object
device: torch.device
# Create minimal mock processor
class MockProcessor:
def apply_chat_template(self, messages, tokenize=False, add_generation_prompt=True):
return "mock template text"
def __call__(self, text=None, images=None, return_tensors=None, padding=None):
# Return mock tensor dict with input_ids
return {
'input_ids': torch.randint(0, 100, (1, 10)),
'attention_mask': torch.ones((1, 10), dtype=torch.long)
}
def batch_decode(self, token_ids, skip_special_tokens=False):
return ["mock generated text response"]
# Create minimal mock model
class MockModel(nn.Module):
def __init__(self):
super().__init__()
self.linear = nn.Linear(4, 4)
def generate(self, input_ids=None, attention_mask=None, do_sample=False,
top_p=None, temperature=None, max_new_tokens=None):
batch_size = input_ids.shape[0]
# Return generated tokens (batch_size, seq_len) - include prompt + new tokens
return torch.randint(0, 100, (batch_size, 15))
# Create mock bundle
processor = MockProcessor()
model = MockModel()
device = torch.device('cpu')
bundle = ModelBundle(processor=processor, model=model, device=device)
# Create sample images
pil_images = [Image.new('RGB', (4, 3)), Image.new('RGB', (4, 3))]
ARGS = {
'bundle': bundle,
'question': 'What is in this image?',
'pil_images': pil_images,
'samples': 3,
'max_new_tokens': 5
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- PIL, numpy, torch, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| strip_media_tokens |
Ran ran; our contract draft was wrong, not the code |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("543eb8006243777b") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import re
MEDIA_TOKEN_RE = re.compile('<image>|<video>|<audio>')
def normalize_text(x: object) -> str:
if x is None:
return ''
return ' '.join(str(x).strip().split())
def strip_media_tokens(text: str) -> str:
return normalize_text(MEDIA_TOKEN_RE.sub(' ', text))
# Setup code
ARGS = {
'text': 'This is a <image> test with <video> and <audio> tokens'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| temporal_regex_score |
Ran ran, honoured the contract we drafted |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("098895353483bfe1") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import re
TEMPORAL_RE = re.compile('\\b(before|after|then|while|during|first|next|finally|sequence|later|earlier|transition|happen)\\b', re.IGNORECASE)
def temporal_regex_score(question: str) -> float:
return 1.0 if TEMPORAL_RE.search(question or '') else 0.0
# Test cases for the function
test_question_with_temporal = "What happens before the event?"
test_question_without_temporal = "What is the color of the sky?"
ARGS = {
"question": test_question_with_temporal
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| text_feature |
Ran ran, honoured the contract we drafted |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("87c30108595426b1") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import re
from typing import Dict, Iterator, List, Optional, Sequence, Tuple
import numpy as np
WORD_RE = re.compile('[A-Za-z0-9_]+')
def tokenize(text: str) -> List[str]:
return WORD_RE.findall((text or '').lower())
def text_feature(question: str, answer: str, bins: int=16) -> List[float]:
vec = np.zeros((bins,), dtype=np.float32)
toks = tokenize(question) + tokenize(answer)
if not toks:
return vec.tolist()
for t in toks:
idx = (hash(t) % bins + bins) % bins
vec[idx] += 1.0
vec /= float(np.linalg.norm(vec) + 1e-06)
return vec.tolist()
ARGS = {
"question": "What is machine learning?",
"answer": "Machine learning is a subset of artificial intelligence.",
"bins": 16
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| tnc_llm |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("995b24926b03d47f") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from dataclasses import dataclass
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration
torch.manual_seed(0)
class MockProcessor:
def apply_chat_template(self, messages, tokenize=False, add_generation_prompt=False):
return "mock prompt text"
def __call__(self, text=None, return_tensors=None):
batch_size = len(text) if text else 1
return {
'input_ids': torch.randint(0, 1000, (batch_size, 4)),
'attention_mask': torch.ones(batch_size, 4)
}
def batch_decode(self, token_ids, skip_special_tokens=False):
return ["temporal"]
class MockQwen3VL(nn.Module):
def __init__(self):
super().__init__()
self.linear = nn.Linear(4, 4)
def generate(self, input_ids=None, attention_mask=None, do_sample=False, max_new_tokens=3, **kwargs):
batch_size = input_ids.shape[0]
seq_len = input_ids.shape[1]
new_seq_len = seq_len + max_new_tokens
return torch.randint(0, 1000, (batch_size, new_seq_len))
@dataclass
class ModelBundle:
processor: AutoProcessor
model: Qwen3VLForConditionalGeneration
device: torch.device
mock_processor = MockProcessor()
mock_model = MockQwen3VL()
mock_device = torch.device('cpu')
bundle = ModelBundle(
processor=mock_processor,
model=mock_model,
device=mock_device
)
question = "Does this video show motion?"
ARGS = {
'bundle': bundle,
'question': question
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| tokenize |
Ran ran; our contract draft was wrong, not the code |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("50bcada7381a2399") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import re
from typing import Dict, Iterator, List, Optional, Sequence, Tuple
WORD_RE = re.compile('[A-Za-z0-9_]+')
def tokenize(text: str) -> List[str]:
return WORD_RE.findall((text or '').lower())
ARGS = {
'text': 'Hello World 123 test_case'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| vision_feature |
Ran ran, honoured the contract we drafted |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("271487e7d8776727") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import tempfile
import os
import numpy as np
import cv2
# Create temporary directory with actual image files
temp_dir = tempfile.mkdtemp()
# Create a few simple test images
image_paths = []
for i in range(5):
img = np.random.randint(0, 255, (10, 10, 3), dtype=np.uint8)
path = os.path.join(temp_dir, f"test_image_{i}.png")
cv2.imwrite(path, img)
image_paths.append(path)
ARGS = {
"paths": image_paths,
"max_frames": 3
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- cv2, numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| ModelBundle |
Not yet run unverified (harvested, no recorded run) |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("ba6e95dac46d63b8") How it ran |
| build_model |
Not yet run unverified (harvested, no recorded run) |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("70149d03ea2ec0e9") How it ran |
| compute_loss |
Not yet run unverified (harvested, no recorded run) |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("a71fb50b5b3bd84e") How it ran |
| main |
Not yet run unverified (harvested, no recorded run) |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("3968b3aa5d9e95e8") How it ran |
| parse_args |
Not yet run unverified (harvested, no recorded run) |
rujiewu/GDO/gdo/extract_six_metrics.py code served (permissive licence) · get_code("85315dad44b0f3f1") How it ran |