| whitespace_tokenize |
Ran ran; our contract draft was wrong, not the code |
clovaai/bros/bros/tokenization_bros.py code served (permissive licence) · get_code("cf9ffa02a42184af") How it ranRan in Syntology's jail on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. The comparison behind this sample's level is a different run: the peer-agreement lane ran it again on the host, not in the jail (report dated 2026-09-05T02:55:57Z). If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
def whitespace_tokenize(text):
"""Runs basic whitespace cleaning and splitting on a piece of text."""
text = text.strip()
if not text:
return []
tokens = text.split()
return tokens
# Setup code
torch.manual_seed(0)
# The function takes a single parameter: text (a string)
# It performs basic whitespace cleaning and splitting
# We need to provide a valid string argument
ARGS = {
"text": " hello world this is a test "
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| do_eval_step |
Ran ran on a synthesized input |
clovaai/bros/lightning_modules/bros_bies_module.py code served (permissive licence) · get_code("3944c09a44634e37") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# Small batch size = 2, sequence length = 3, num_classes = 4
batch_size = 2
max_seq_length = 3
num_classes = 4
# Create batch dict with labels tensor
# Labels shape: (batch_size, max_seq_length)
# Values: class indices or ignore_index (-100)
labels = torch.tensor([
[0, 1, 2],
[1, -100, 0]
], dtype=torch.long)
batch = {
'labels': labels
}
# Create head_outputs: logits of shape (batch_size, max_seq_length, num_classes)
head_outputs = torch.randn(batch_size, max_seq_length, num_classes)
# Create a scalar loss value
loss = torch.tensor(0.5)
# Create eval_kwargs with ignore_index and label_map
eval_kwargs = {
'ignore_index': -100,
'label_map': {
0: 'O',
1: 'B-PER',
2: 'I-PER',
3: 'B-LOC'
}
}
ARGS = {
'batch': batch,
'head_outputs': head_outputs,
'loss': loss,
'eval_kwargs': eval_kwargs
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| do_eval_step |
Ran ran on a synthesized input |
clovaai/bros/lightning_modules/bros_bio_module.py code served (permissive licence) · get_code("b0682ddacb993e88") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Build tiny tensors for the batch
batch_size = 2
seq_len = 4
num_classes = 3
# head_outputs: logits of shape (batch_size, seq_len, num_labels)
# num_labels = 2 * num_classes + 1 (for O, B_class, I_class for each class)
num_labels = 2 * num_classes + 1
head_outputs = torch.randn(batch_size, seq_len, num_labels)
# batch dict with required fields
batch = {
'bio_labels': torch.randint(0, num_labels, (batch_size, seq_len)),
'are_box_first_tokens': torch.tensor([
[True, True, False, False],
[True, False, True, False]
])
}
# loss value
loss = torch.tensor(0.5)
# eval_kwargs with class_names
class_names = ['class_0', 'class_1', 'class_2']
eval_kwargs = {
'class_names': class_names
}
ARGS = {
'batch': batch,
'head_outputs': head_outputs,
'loss': loss,
'eval_kwargs': eval_kwargs
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| do_eval_step |
Ran ran on a synthesized input |
clovaai/bros/lightning_modules/bros_spade_module.py code served (permissive licence) · get_code("a19d5a58a6632b6a") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# Create minimal batch dictionary
batch = {
'itc_labels': torch.tensor([[1, 2, 0, 1], [2, 1, 0, 0]], dtype=torch.long), # (bsz=2, seq_len=4)
'are_box_first_tokens': torch.tensor([[True, False, False, True], [True, True, False, False]], dtype=torch.bool), # (bsz=2, seq_len=4)
'stc_labels': torch.tensor([[0, 1, 0, 2], [0, 2, 0, 1]], dtype=torch.long), # (bsz=2, seq_len=4)
'attention_mask': torch.tensor([[1, 1, 1, 1], [1, 1, 0, 1]], dtype=torch.long), # (bsz=2, seq_len=4)
}
# Create head outputs with logits for ITC and STC
head_outputs = {
'itc_outputs': torch.randn(2, 4, 3), # (bsz=2, seq_len=4, num_classes=3)
'stc_outputs': torch.randn(2, 4, 5), # (bsz=2, seq_len=4, max_seq=5)
}
# Scalar loss value
loss = torch.tensor(0.5)
# Create eval_kwargs
eval_kwargs = {
'class_names': ['dummy', 'class_a', 'class_b'], # 3 classes, 0 is dummy
'dummy_idx': 0,
}
ARGS = {
'batch': batch,
'head_outputs': head_outputs,
'loss': loss,
'eval_kwargs': eval_kwargs,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| do_eval_step |
Ran ran on a synthesized input |
clovaai/bros/lightning_modules/bros_spade_rel_module.py code served (permissive licence) · get_code("a24cbc913ce1517c") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# Create a small batch with tensors
batch_size = 2
seq_length = 4
num_classes = 5
dummy_idx = 0
# Create el_outputs (logits) for argmax operation
# Shape: (batch_size, seq_length, num_classes)
el_outputs = torch.randn(batch_size, seq_length, num_classes)
# Create head_outputs dictionary containing el_outputs
head_outputs = {
'el_outputs': el_outputs
}
# Create ground truth el_labels
# Shape: (batch_size, seq_length)
gt_el_labels = torch.tensor([
[0, 1, 2, 0],
[0, 0, 3, 1]
], dtype=torch.long)
# Create are_box_first_tokens mask
# Shape: (batch_size, seq_length) - binary mask indicating first tokens of boxes
are_box_first_tokens = torch.tensor([
[0, 1, 1, 0],
[0, 1, 1, 1]
], dtype=torch.long)
# Create batch dictionary
batch = {
'el_labels': gt_el_labels,
'are_box_first_tokens': are_box_first_tokens
}
# Create loss value
loss = torch.tensor(0.5, dtype=torch.float32)
# Create eval_kwargs dictionary
eval_kwargs = {
'dummy_idx': dummy_idx
}
ARGS = {
'batch': batch,
'head_outputs': head_outputs,
'loss': loss,
'eval_kwargs': eval_kwargs
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| eval_ee_bio_batch |
Ran ran on a synthesized input |
clovaai/bros/lightning_modules/bros_bio_module.py code served (permissive licence) · get_code("62ed443571cf13ef") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# Understand the function signature:
# eval_ee_bio_batch(pr_labels, gt_labels, are_box_first_tokens, class_names)
#
# pr_labels: tensor indexed as pr_labels[example_idx][token_idx] -> used with .item()
# gt_labels: tensor indexed as gt_labels[example_idx][token_idx] -> used with .item()
# are_box_first_tokens: tensor indexed as are_box_first_tokens[example_idx] -> used as mask
# class_names: list used in len(class_names) and range(len(class_names))
# Create small tensors:
# batch_size = 2, seq_length = 4, num_classes = 3
batch_size = 2
seq_length = 4
num_classes = 3
# pr_labels and gt_labels: shape (batch_size, seq_length)
# Values are label IDs in BIO format:
# 0 = O (other)
# 1, 2 = B, I tags for class 0
# 3, 4 = B, I tags for class 1
# 5, 6 = B, I tags for class 2
pr_labels = torch.tensor([
[0, 1, 2, 3], # example 0: O, B-class0, I-class0, B-class1
[1, 1, 0, 2] # example 1: B-class0, B-class0, O, I-class0
], dtype=torch.long)
gt_labels = torch.tensor([
[0, 1, 2, 2], # example 0: O, B-class0, I-class0, I-class0
[1, 2, 0, 0] # example 1: B-class0, I-class0, O, O
], dtype=torch.long)
# are_box_first_tokens: shape (batch_size, seq_length), boolean mask
# True where a token should be considered
are_box_first_tokens = torch.tensor([
[True, True, True, True],
[True, True, False, True]
], dtype=torch.bool)
# class_names: list of class names
class_names = ["class0", "class1", "class2"]
ARGS = {
"pr_labels": pr_labels,
"gt_labels": gt_labels,
"are_box_first_tokens": are_box_first_tokens,
"class_names": class_names
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family int — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| eval_ee_bio_example |
Ran ran on a synthesized input |
clovaai/bros/lightning_modules/bros_bio_module.py code served (permissive licence) · get_code("1bfd47528da7b59e") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# Analyze the function usage:
# - pr_seq: indexed with [], items called .item() → torch.Tensor of label IDs
# - gt_seq: indexed with [], items called .item() → torch.Tensor of label IDs
# - box_first_token_mask: used for boolean indexing → torch.Tensor of bools
# - class_names: len() called → list or tuple of class names
# BIO tagging scheme:
# - label_id 0 = no tag (skipped)
# - label_id 1, 3, 5, ... = B tags (odd, for classes 0, 1, 2, ...)
# - label_id 2, 4, 6, ... = I tags (even, for classes 0, 1, 2, ...)
# So we need at least 2 * num_classes label IDs
# Create small sequences with valid BIO labels
num_classes = 3
seq_length = 8
# Ground truth sequence: mix of B and I tags for different classes
# label_id: 0=O, 1=B-class0, 2=I-class0, 3=B-class1, 4=I-class1, 5=B-class2, 6=I-class2
gt_seq = torch.tensor([0, 1, 2, 2, 3, 4, 0, 5], dtype=torch.long)
# Prediction sequence: some correct, some incorrect
pr_seq = torch.tensor([0, 1, 2, 0, 3, 4, 5, 6], dtype=torch.long)
# Boolean mask: which positions are valid (first tokens of boxes)
box_first_token_mask = torch.tensor([True, True, True, False, True, True, False, True], dtype=torch.bool)
# Class names
class_names = ["PER", "ORG", "LOC"]
ARGS = {
"pr_seq": pr_seq,
"gt_seq": gt_seq,
"box_first_token_mask": box_first_token_mask,
"class_names": class_names,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family int — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| eval_ee_spade_batch |
Ran ran on a synthesized input |
clovaai/bros/lightning_modules/bros_spade_module.py code served (permissive licence) · get_code("0e5e7de5c6b14651") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# Define class names
class_names = ["Person", "Organization", "Location"]
# Batch size and sequence length
batch_size = 2
seq_len = 4
# Create tensor arguments
# pr_itc_labels and gt_itc_labels: shape (batch_size, seq_len), values in range [0, len(class_names))
pr_itc_labels = torch.randint(0, len(class_names), (batch_size, seq_len))
gt_itc_labels = torch.randint(0, len(class_names), (batch_size, seq_len))
# are_box_first_tokens: shape (batch_size, seq_len), boolean mask
are_box_first_tokens = torch.tensor([
[True, False, True, False],
[True, True, False, False]
], dtype=torch.bool)
# pr_stc_labels and gt_stc_labels: shape (batch_size, seq_len), values representing next token indices
# dummy_idx is used as a special value to indicate no connection
dummy_idx = 0
pr_stc_labels = torch.tensor([
[0, 1, 0, 2],
[0, 2, 0, 0]
], dtype=torch.long)
gt_stc_labels = torch.tensor([
[0, 1, 0, 2],
[0, 2, 0, 0]
], dtype=torch.long)
# attention_mask: shape (batch_size, seq_len), 1 for valid tokens, 0 for padding
attention_mask = torch.tensor([
[1, 1, 1, 1],
[1, 1, 1, 0]
], dtype=torch.long)
ARGS = {
"pr_itc_labels": pr_itc_labels,
"gt_itc_labels": gt_itc_labels,
"are_box_first_tokens": are_box_first_tokens,
"pr_stc_labels": pr_stc_labels,
"gt_stc_labels": gt_stc_labels,
"attention_mask": attention_mask,
"class_names": class_names,
"dummy_idx": dummy_idx
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family int — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| eval_ee_spade_example |
Ran ran on a synthesized input |
clovaai/bros/lightning_modules/bros_spade_module.py code served (permissive licence) · get_code("cf68c6ab08d0c012") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# Create small tensor arguments
# Tensors need to be on CPU and support .cpu().numpy()
batch_size = 5
num_classes = 3
# pr_itc_label: predicted initial token class labels
# Shape: (num_tokens,) with values in [0, num_classes)
pr_itc_label = torch.tensor([0, 1, 2, 1, 0], dtype=torch.long)
# gt_itc_label: ground truth initial token class labels
gt_itc_label = torch.tensor([0, 1, 2, 1, 0], dtype=torch.long)
# box_first_token_mask: boolean mask indicating first tokens of boxes
# Shape: (num_tokens,) with boolean values
box_first_token_mask = torch.tensor([True, True, True, False, True], dtype=torch.bool)
# pr_stc_label: predicted subsequent token class labels
# Contains indices pointing to next tokens or dummy_idx
pr_stc_label = torch.tensor([0, 2, 3, 0, 0], dtype=torch.long)
# gt_stc_label: ground truth subsequent token class labels
gt_stc_label = torch.tensor([0, 2, 3, 0, 0], dtype=torch.long)
# attention_mask: mask indicating valid tokens
# Shape: (num_tokens,) with boolean values
attention_mask = torch.tensor([True, True, True, True, True], dtype=torch.bool)
# class_names: list of class name strings
class_names = ["class_0", "class_1", "class_2"]
# dummy_idx: special index used to mark end of sequences
dummy_idx = 0
ARGS = {
"pr_itc_label": pr_itc_label,
"gt_itc_label": gt_itc_label,
"box_first_token_mask": box_first_token_mask,
"pr_stc_label": pr_stc_label,
"gt_stc_label": gt_stc_label,
"attention_mask": attention_mask,
"class_names": class_names,
"dummy_idx": dummy_idx,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family int — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| eval_el_spade_batch |
Ran ran on a synthesized input |
clovaai/bros/lightning_modules/bros_spade_rel_module.py code served (permissive licence) · get_code("ffb90147092f9680") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# Create small tensors for testing
batch_size = 2
seq_length = 4
# pr_el_labels: predicted entity link labels, shape (batch_size, seq_length)
# Values should be entity indices or dummy_idx or 0
pr_el_labels = torch.tensor([
[1, 2, 0, 0],
[1, 0, 2, 0]
], dtype=torch.long)
# gt_el_labels: ground truth entity link labels, shape (batch_size, seq_length)
gt_el_labels = torch.tensor([
[1, 2, 0, 0],
[1, 0, 2, 3]
], dtype=torch.long)
# are_box_first_tokens: mask indicating which tokens are first tokens of boxes
# shape (batch_size, seq_length), binary values (0 or 1)
are_box_first_tokens = torch.tensor([
[1, 1, 0, 0],
[1, 0, 1, 1]
], dtype=torch.long)
# dummy_idx: index used to mark dummy/padding tokens
dummy_idx = 0
ARGS = {
"pr_el_labels": pr_el_labels,
"gt_el_labels": gt_el_labels,
"are_box_first_tokens": are_box_first_tokens,
"dummy_idx": dummy_idx,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family int — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| eval_el_spade_example |
Ran ran on a synthesized input |
clovaai/bros/lightning_modules/bros_spade_rel_module.py code served (permissive licence) · get_code("8e888f85f35bcb89") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# Create small tensors for testing
# pr_el_label: predicted element labels - shape (seq_len,)
pr_el_label = torch.tensor([0, 1, 2, 1, 0, 3], dtype=torch.long)
# gt_el_label: ground truth element labels - shape (seq_len,)
gt_el_label = torch.tensor([0, 1, 2, 1, 0, 3], dtype=torch.long)
# box_first_token_mask: binary mask indicating first token of each box - shape (seq_len,)
box_first_token_mask = torch.tensor([1, 1, 1, 0, 1, 1], dtype=torch.long)
# dummy_idx: index used for padding/dummy tokens
dummy_idx = 0
ARGS = {
"pr_el_label": pr_el_label,
"gt_el_label": gt_el_label,
"box_first_token_mask": box_first_token_mask,
"dummy_idx": dummy_idx,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family int — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_label_map |
Ran ran on a synthesized input |
clovaai/bros/lightning_modules/bros_bies_module.py code served (permissive licence) · get_code("07564a0cc69b377b") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import os
import tempfile
# Create a temporary directory and a labels.txt file
temp_dir = tempfile.mkdtemp()
labels_file = os.path.join(temp_dir, 'labels.txt')
with open(labels_file, 'w', encoding='utf-8') as f:
f.write('cat\n')
f.write('dog\n')
f.write('bird\n')
ARGS = {
'dataset_root_path': temp_dir
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| load_vocab |
Ran ran; our contract draft was wrong, not the code |
clovaai/bros/bros/tokenization_bros.py code served (permissive licence) · get_code("e7fbc7a74a3457c7") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned OrderedDict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import collections
import tempfile
import os
# Create a temporary vocabulary file
temp_dir = tempfile.mkdtemp()
vocab_file = os.path.join(temp_dir, 'vocab.txt')
# Write some sample tokens to the vocab file
sample_tokens = ['[PAD]', '[UNK]', '[CLS]', '[SEP]', 'the', 'quick', 'brown', 'fox']
with open(vocab_file, 'w', encoding='utf-8') as f:
for token in sample_tokens:
f.write(token + '\n')
ARGS = {
'vocab_file': vocab_file
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type OrderedDict — a OrderedDict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a OrderedDict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|