| compute_score_with_logits |
Ran ran; our contract draft was wrong, not the code |
dreichCSL/TrueVG/components/losses.py code served (permissive licence) · get_code("89bdd6decba2b415") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# The function expects:
# 1. logits: a tensor that can be passed to torch.max(logits, 1) - needs shape (batch_size, num_classes)
# 2. labels: a tensor that has .size() method and supports scatter_ operations - needs same shape as logits for element-wise multiplication
batch_size = 2
num_classes = 4
# Create logits tensor with shape (batch_size, num_classes)
logits = torch.randn(batch_size, num_classes)
# Create labels tensor with shape (batch_size, num_classes)
# This should be one-hot encoded or binary labels for the scores computation to make sense
labels = torch.zeros(batch_size, num_classes)
labels[0, 1] = 1
labels[1, 3] = 1
ARGS = {
"logits": logits,
"labels": labels
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| find_infusion_objects |
Ran ran; our contract draft was wrong, not the code |
dreichCSL/TrueVG/preprocessing/truevg/prep_infusion.py code served (permissive licence) · get_code("69f8cd8f0aee84d4") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import pickle
import json
import h5py
import numpy as np
import tempfile
import os
# Create a temporary directory for test data
temp_dir = tempfile.mkdtemp()
preprocessing_dir = os.path.join(temp_dir, 'preprocessing')
os.makedirs(preprocessing_dir, exist_ok=True)
# Create minimal GQA reference JSON
reference_gqa_json = {
'img_001': {'index': 0, 'objectsNum': 3},
'img_002': {'index': 1, 'objectsNum': 2},
}
with open(os.path.join(temp_dir, 'output_gqa_detectron_objects_info.json'), 'w') as f:
json.dump(reference_gqa_json, f)
# Create object name classes pickle
objectname_classes = {
'dog': 0,
'cat': 1,
'person': 2,
'car': 3,
'chair': 4,
}
with open(os.path.join(preprocessing_dir, 'objectname_classes.pkl'), 'wb') as f:
pickle.dump(objectname_classes, f)
# Create attribute categories pickle
attribute_categories = {
'color': ['red', 'blue', 'green'],
'size': ['small', 'large'],
}
with open(os.path.join(preprocessing_dir, 'attribute_categories.pkl'), 'wb') as f:
pickle.dump(attribute_categories, f)
# Create HDF5 image database
with h5py.File(os.path.join(preprocessing_dir, 'img_db.h5'), 'w') as hf:
# Create datasets for object names and attributes
# img_index 0: 3 objects; img_index 1: 2 objects
name_data = np.array([[0, 1, 2], [3, 4, -1]], dtype=np.int32)
hf.create_dataset('name', data=name_data)
# Attributes: -1 means no attributes, otherwise indices into attribute categories
attr_data = np.array([
[[0, -1], [1, 0], [0, 1]], # img_index 0: 3 objects with attributes
[[1, 1], [0, -1], [-1, -1]], # img_index 1: 2 objects
], dtype=np.int32)
hf.create_dataset('attributes', data=attr_data)
# Create scene graph (sg)
sg = {
'img_001': {
'objects': {
0: {'name': 'dog', 'attributes': ['red', 'small']},
1: {'name': 'cat', 'attributes': []},
2: {'name': 'person', 'attributes': ['large']},
}
},
'img_002': {
'objects': {
0: {'name': 'car', 'attributes': ['blue']},
1: {'name': 'chair', 'attributes': ['red', 'large']},
}
},
}
# Create QA dict
qa = {
'q_001': {
'imageId': 'img_001',
'annotations': {
'answer': {0: 0, 1: 1},
'fullAnswer': {0: 0},
'question': {0: 2},
}
},
'q_002': {
'imageId': 'img_002',
'annotations': {
'answer': {0: 0},
'fullAnswer': {0: 1},
'question': {0: 0},
}
},
}
# Create relevant_objects_per_img_and_question
relevant_objects_per_img_and_question = {
'img_001': {
'q_001': {
0: [0, 1, 2],
1: [0, 1],
2: [0, 2],
}
},
'img_002': {
'q_002': {
0: [0, 1],
1: [1],
}
},
}
ARGS = {
'sg': sg,
'qa': qa,
'relevant_objects_per_img_and_question': relevant_objects_per_img_and_question,
'data_dir': temp_dir,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- h5py, numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_FIscores_INF_semantic_matching |
Ran ran; our contract draft was wrong, not the code |
dreichCSL/TrueVG/preprocessing/truevg/prep_hints.py code served (permissive licence) · get_code("ed7ea0c3706614fe") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import os
import numpy as np
import json
import tempfile
# Create a temporary directory for test data
temp_dir = tempfile.mkdtemp()
# Create the required JSON file with GQA object info
gqa_data = {
"img_1": {"objectsNum": 10},
"img_2": {"objectsNum": 15},
"img_3": {"objectsNum": 8},
}
json_path = os.path.join(temp_dir, 'output_gqa_detectron_objects_info.json')
with open(json_path, 'w') as f:
json.dump(gqa_data, f)
# Build qa dict: maps question_id to question info with imageId
qa = {
"q_1": {"imageId": "img_1", "question": "What is this?"},
"q_2": {"imageId": "img_2", "question": "How many?"},
"q_3": {"imageId": "img_3", "question": "Where?"},
}
# Build rel_obj dict: maps imageId -> qid -> list of relevant objects
rel_obj = {
"img_1": {
"q_1": [1, 2, 3],
},
"img_2": {
"q_2": [5, 6],
},
"img_3": {
"q_3": [0, 1],
},
}
# Build infusion_database: maps imageId -> qid -> objid -> list of boxes
# Each box is a tuple (box_index, box_info)
infusion_database = {
"img_1": {
"q_1": {
"obj_1": [(1, {"x": 10, "y": 20}), (2, {"x": 30, "y": 40})],
"obj_2": [(3, {"x": 50, "y": 60})],
}
},
"img_2": {
"q_2": {
"obj_5": [(5, {"x": 100, "y": 110})],
}
},
"img_3": {
"q_3": {
"obj_0": [(0, {"x": 5, "y": 15})],
}
},
}
# Build arguments dict
ARGS = {
"qa": qa,
"rel_obj": rel_obj,
"infusion_database": infusion_database,
"data_dir": temp_dir,
"max_objects": 100,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_FIscores_INF_spatial_matching |
Ran ran; our contract draft was wrong, not the code |
dreichCSL/TrueVG/preprocessing/truevg/prep_hints.py code served (permissive licence) · get_code("bbab318c93ed2fb0") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import os
import json
import tempfile
import torch
import numpy as np
# Create a temporary directory for data
temp_dir = tempfile.mkdtemp()
# Create the required GQA detectron objects info JSON file
gqa_data = {
"img_001": {"objectsNum": 50},
"img_002": {"objectsNum": 30},
}
output_path = os.path.join(temp_dir, 'output_gqa_detectron_objects_info.json')
with open(output_path, 'w') as f:
json.dump(gqa_data, f)
# Create qa dict with image IDs
qa = {
"q_001": {"imageId": "img_001"},
"q_002": {"imageId": "img_002"},
"q_003": {"imageId": "img_001"},
}
# Create FIscores_spatial dict with numpy arrays indexed by question ID
torch.manual_seed(0)
FIscores_spatial = {
"q_001": np.random.randn(100),
"q_002": np.random.randn(100),
"q_003": np.random.randn(100),
}
# Create infusion_database dict
# Structure: {img_id: {qid: {objid: [list of (box_idx, box_info) tuples]}}}
infusion_database = {
"img_001": {
"q_001": {
"obj_1": [(5, "box_info_1"), (10, "box_info_2")],
"obj_2": [(15, "box_info_3")],
},
"q_003": {
"obj_3": [(100, "box_info_4")],
},
},
"img_002": {
"q_002": {
"obj_4": [(8, "box_info_5")],
},
},
}
# Define ARGS dictionary
ARGS = {
"qa": qa,
"FIscores_spatial": FIscores_spatial,
"infusion_database": infusion_database,
"data_dir": temp_dir,
"max_objects": 100,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_iou |
Ran ran, honoured the contract we drafted |
dreichCSL/TrueVG/preprocessing/truevg/prep_evaluation.py code served (permissive licence) · get_code("47aac06436736844") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
# Setup code for get_iou function
# Define two bounding boxes as dictionaries
# bb1: bounding box 1 with coordinates (x1, y1) at top-left and (x2, y2) at bottom-right
bb1 = {
'x1': 10,
'y1': 20,
'x2': 50,
'y2': 80
}
# bb2: bounding box 2 with coordinates (x1, y1) at top-left and (x2, y2) at bottom-right
bb2 = {
'x1': 30,
'y1': 40,
'x2': 70,
'y2': 100
}
ARGS = {
'bb1': bb1,
'bb2': bb2
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_overlap |
Ran ran, honoured the contract we drafted |
dreichCSL/TrueVG/preprocessing/truevg/prep_evaluation.py code served (permissive licence) · get_code("77c2c6aa4beabb9b") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
# Setup code for get_overlap function
# Define bounding boxes as dictionaries with x1, x2, y1, y2 keys
# bb1: object bounding box
bb1 = {
'x1': 10,
'y1': 20,
'x2': 50,
'y2': 60
}
# bb2: area bounding box that overlaps with bb1
bb2 = {
'x1': 30,
'y1': 40,
'x2': 70,
'y2': 80
}
ARGS = {
'bb1': bb1,
'bb2': bb2
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_prediction_scores |
Ran ran; our contract draft was wrong, not the code |
dreichCSL/TrueVG/components/losses.py code served (permissive licence) · get_code("1a250cde60dc9077") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family int, shape (2, 1) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# Construct logits: shape (batch_size=2, num_classes=4)
# After torch.max(logits, 1)[1], we get indices of shape (2,)
logits = torch.randn(2, 4)
# Construct labels: shape (batch_size=2, num_classes=4)
# The function does labels.gather(1, logits.unsqueeze(1))
# logits.unsqueeze(1) will have shape (2, 1) after max operation
# gather(1, indices) with indices shape (2, 1) requires labels to have compatible shape
labels = torch.randint(0, 10, (2, 4))
ARGS = {
"logits": logits,
"labels": labels,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 2, dtype family int, shape (2, 1) — shape (2, 1) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_truevg_qids |
Ran ran; our contract draft was wrong, not the code |
dreichCSL/TrueVG/preprocessing/truevg/prep_evaluation.py code served (permissive licence) · get_code("f314ec8478f617c7") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
# Construct nested dictionary structures for the function parameters
# Based on the function logic:
# - rel_obj: dict[imgid] -> dict[qid] -> list (non-empty lists)
# - irrel_obj: dict[imgid] -> dict[qid] -> list (non-empty lists)
# - infusion_database: dict[imgid] -> dict[qid] -> dict[objid] -> list of boxes
# where each box is [something, [class_id, ...], ...]
# Create rel_obj structure
rel_obj = {
"img_1": {
"q_1": [1, 2, 3],
"q_2": [4, 5],
},
"img_2": {
"q_3": [6],
}
}
# Create irrel_obj structure
irrel_obj = {
"img_1": {
"q_1": [7, 8],
"q_3": [9],
},
"img_2": {
"q_2": [10],
"q_4": [11],
}
}
# Create infusion_database structure
# box format: [index, [class_id, ...], ...]
# Boxes with class_id != 11 or (class_id == 11 and index == 100) will be included in infused_qids
infusion_database = {
"img_1": {
"q_1": {
"obj_1": [
[50, [5, 6]],
[100, [11, 12]],
],
},
},
"img_2": {
"q_3": {
"obj_2": [
[75, [11, 13]],
],
},
}
}
ARGS = {
"rel_obj": rel_obj,
"irrel_obj": irrel_obj,
"infusion_database": infusion_database,
"fpvg_mode": True,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| instance_bce_with_logits |
Ran ran; our contract draft was wrong, not the code |
dreichCSL/TrueVG/components/losses.py code served (permissive licence) · get_code("2b00263b6afeb3ee") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Create a minimal config object with vqa_loss_type attribute
class OptConfig:
def __init__(self, vqa_loss_type):
self.vqa_loss_type = vqa_loss_type
# Test with sigmoid loss type
opt_sigmoid = OptConfig(vqa_loss_type='sigmoid')
# Create small tensors: batch_size=2, num_answers=4
batch_size = 2
num_answers = 4
# Logits: (batch_size, num_answers)
logits = torch.randn(batch_size, num_answers)
# Labels for sigmoid: (batch_size, num_answers) with values in [0, 1]
labels = torch.randint(0, 2, (batch_size, num_answers)).float()
ARGS = {
'opt': opt_sigmoid,
'logits': logits,
'labels': labels
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|