| add2list_p |
Ran ran; our contract draft was wrong, not the code |
chongli17/crosslingualalignment/src/generate_sup_data.py code served (permissive licence) · get_code("72bf5e3daa978af0") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import random
# Set seed for reproducibility
random.seed(0)
# Construct valid arguments for add2list_p function
keys = [
{
'id': 'item_1',
'instruction': 'do something',
'field_a': 'value_a_1',
'field_b': 'value_b_1',
},
{
'id': 'item_1',
'instruction': 'do something else',
'field_a': 'value_a_2',
'field_b': 'value_b_2',
}
]
data_list = []
reverse_prob = 0.3
ARGS = {
'keys': keys,
'data_list': data_list,
'reverse_prob': reverse_prob,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple — a tuple output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| avg_list |
Ran ran, violated the contract we drafted |
chongli17/crosslingualalignment/src/trainers.py code served (permissive licence) · get_code("30d3b444d7a9af9f") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# The function avg_list takes a list and computes the average
# It uses: sum(l_in) to add elements, len(l_in) to get length, and float() conversion
# The list elements need to support addition and division
# Create a simple list of numbers
l_in = [1.0, 2.0, 3.0, 4.0, 5.0]
ARGS = {
"l_in": l_in
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| batch_prompts_inference_choices |
Ran ran, honoured the contract we drafted |
chongli17/crosslingualalignment/src/trainers.py code served (permissive licence) · get_code("099c24f9f0ef7fc7") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family int; no output shape is stated. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
class MockTokenizer:
def __init__(self):
self.eos_token = "[EOS]"
self.pad_token = "[PAD]"
def __call__(self, text, padding='longest', return_tensors='pt'):
# Return mock tokenized output with input_ids and attention_mask
batch_size = len(text)
seq_len = 8
input_ids = torch.randint(0, 100, (batch_size, seq_len))
attention_mask = torch.ones((batch_size, seq_len), dtype=torch.long)
return {
'input_ids': input_ids,
'attention_mask': attention_mask
}
class MockModel(nn.Module):
def __init__(self):
super().__init__()
self.linear = nn.Linear(4, 100) # vocab_size = 100
def clm_forward(self, input_ids, attention_mask, return_dict=True):
batch_size, seq_len = input_ids.shape
logits = self.linear(torch.randn(batch_size, seq_len, 4)) # (batch_size, seq_len, vocab_size)
class Output:
def __init__(self, logits):
self.logits = logits
return Output(logits=logits)
model = MockModel()
tokenizer = MockTokenizer()
sample_dicts = [
{
'demos': 'Example: ',
'premise': 'The cat sat on the mat',
'hypothesis': 'An animal is on furniture'
},
{
'demos': 'Example: ',
'premise': 'It is raining outside',
'hypothesis': 'The weather is sunny'
}
]
prompt_style = '{demos}{premise},right?{choice},{hypothesis}'
choice_strs = [
['entailment', 'neutral', 'contradiction'],
['entailment', 'neutral', 'contradiction']
]
ARGS = {
'model': model,
'tokenizer': tokenizer,
'sample_dicts': sample_dicts,
'prompt_style': prompt_style,
'choice_strs': choice_strs,
'cuda': False
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family int — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| flatten_in |
Ran ran; our contract draft was wrong, not the code |
chongli17/crosslingualalignment/src/decoder_models.py code served (permissive licence) · get_code("455ae9e7f7997cfc") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned OutputDict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Create a minimal encoder that mimics HuggingFace transformer behavior
class MinimalEncoder(nn.Module):
def __init__(self, hidden_size=4, vocab_size=100):
super().__init__()
self.embedding = nn.Embedding(vocab_size, hidden_size)
self.linear = nn.Linear(hidden_size, hidden_size)
def forward(self, input_ids, attention_mask=None, token_type_ids=None,
output_hidden_states=False, return_dict=True):
# Simple embedding pass
hidden_states = self.embedding(input_ids)
hidden_states = self.linear(hidden_states)
# Return dict with expected keys
if return_dict:
class OutputDict:
def __init__(self, last_hidden_state, hidden_states=None):
self.last_hidden_state = last_hidden_state
self.hidden_states = hidden_states
if output_hidden_states:
return OutputDict(hidden_states, hidden_states=[hidden_states])
else:
return OutputDict(hidden_states)
return hidden_states
encoder = MinimalEncoder(hidden_size=4, vocab_size=100)
# Create input tensors with batch dimension that can be flattened
# Shape: (batch_size=2, num_sentences=3, seq_length=5)
# After flattening: (6, 5)
input_ids = torch.randint(0, 100, (2, 3, 5))
attention_mask = torch.ones((2, 3, 5), dtype=torch.long)
token_type_ids = torch.zeros((2, 3, 5), dtype=torch.long)
ARGS = {
"encoder": encoder,
"input_ids": input_ids,
"attention_mask": attention_mask,
"token_type_ids": token_type_ids,
"output_hidden_states": False
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type OutputDict — a OutputDict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a OutputDict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_prompt_format |
Ran ran; our contract draft was wrong, not the code |
chongli17/crosslingualalignment/src/trainers.py code served (permissive licence) · get_code("2bb8b7594e127350") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import json
import tempfile
import os
# Create a temporary directory and a JSON file with prompt formats
temp_dir = tempfile.mkdtemp()
prompt_formats_file = os.path.join(temp_dir, "prompt_formats.json")
# Create sample prompt formats structure
prompt_formats = {
"default": {
"en": "This is an English prompt format",
"fr": "Ceci est un format d'invite en français"
},
"custom": {
"en": "Custom English prompt",
"es": "Aviso personalizado en español"
}
}
with open(prompt_formats_file, 'w') as f:
json.dump(prompt_formats, f)
ARGS = {
'prompt_formats_path': prompt_formats_file,
'prompt_formats_name': 'default',
'prompt_language': 'en'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_random_item |
Ran ran, honoured the contract we drafted |
chongli17/crosslingualalignment/src/generate_sup_data.py code served (permissive licence) · get_code("a7bcb37f1e8879a5") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import random
# The function get_random_item takes a list and returns a random item from it
# We need to construct a simple list with some distinct items
test_list = [
"apple",
"banana",
"cherry",
42,
3.14,
{"key": "value"},
[1, 2, 3]
]
ARGS = {
"in_list": test_list
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_random_items_no_replace |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
chongli17/crosslingualalignment/src/paws_x_prompt.py code served (permissive licence) · get_code("f8dc24ef54ba330b") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import random
# Construct arguments for get_random_items_no_replace
# The function takes:
# - in_list: a list to sample from
# - num: number of items to return (default 1000)
# Create a list with more than 1000 items so num=1000 is valid
in_list = list(range(2000))
# Set num to a valid value less than or equal to len(in_list)
num = 1000
ARGS = {
"in_list": in_list,
"num": num,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family int — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| iterate_in |
Ran ran; our contract draft was wrong, not the code |
chongli17/crosslingualalignment/src/decoder_models.py code served (permissive licence) · get_code("6dca7ca4ad3e5ce8") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Output. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Create a minimal encoder that mimics a BERT-like model
class MinimalEncoder(nn.Module):
def __init__(self, hidden_size=4):
super().__init__()
self.hidden_size = hidden_size
self.linear = nn.Linear(4, hidden_size) # vocab_size implicit as 4
def forward(self, input_ids, attention_mask=None, token_type_ids=None,
output_hidden_states=False, return_dict=False):
# input_ids shape: (batch_size, seq_len)
batch_size, seq_len = input_ids.shape
# Embed and process
embeddings = self.linear(torch.randn(batch_size, seq_len, 4))
last_hidden = embeddings
# Create hidden states if requested
hidden_states = None
if output_hidden_states:
hidden_states = (embeddings, embeddings)
# Create output object with required attributes
class Output:
pass
output = Output()
output.last_hidden_state = last_hidden
output.hidden_states = hidden_states
return output
# Create encoder instance
encoder = MinimalEncoder(hidden_size=4)
# Create input tensors
# Shape: (batch_size=2, num_sent=3, sent_len=5)
batch_size = 2
num_sent = 3
sent_len = 5
hidden_size = 4
input_ids = torch.randint(0, 10, (batch_size, num_sent, sent_len))
attention_mask = torch.ones(batch_size, num_sent, sent_len)
token_type_ids = torch.zeros(batch_size, num_sent, sent_len, dtype=torch.long)
ARGS = {
'encoder': encoder,
'input_ids': input_ids,
'attention_mask': attention_mask,
'token_type_ids': token_type_ids,
'output_hidden_states': True
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Output — a Output output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a Output output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| read_prompt |
Ran ran; our contract draft was wrong, not the code |
chongli17/crosslingualalignment/src/paws_x_prompt.py code served (permissive licence) · get_code("3b1e6bd407417bb8") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import tempfile
import os
# Create a temporary file with some test prompts
temp_file = tempfile.NamedTemporaryFile(mode='w', delete=False, suffix='.txt')
temp_file.write("This is the first prompt\n")
temp_file.write("This is the second prompt\n")
temp_file.write("This is the third prompt\n")
temp_file.close()
ARGS = {
"file_path": temp_file.name
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| read_tsv_data |
Ran ran; our contract draft was wrong, not the code |
chongli17/crosslingualalignment/src/generate_sup_data.py code served (permissive licence) · get_code("0eff3bae8fd00d8d") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import tempfile
import os
# Create a temporary TSV file for testing
temp_dir = tempfile.mkdtemp()
tsv_path = os.path.join(temp_dir, "test_data.tsv")
# Write sample TSV data
with open(tsv_path, 'w', encoding='utf-8') as f:
f.write("col1\tcol2\tcol3\n")
f.write("value1\tvalue2\tvalue3\n")
f.write("data1\tdata2\tdata3\n")
ARGS = {
"tsv_path": tsv_path
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 2, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| read_xnli_tsv |
Ran ran; our contract draft was wrong, not the code |
chongli17/crosslingualalignment/src/xnli_prompt.py code served (permissive licence) · get_code("7ec5fc874e53ea4e") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import tempfile
import os
# Create a temporary TSV file with sample XNLI data
temp_dir = tempfile.mkdtemp()
filepath = os.path.join(temp_dir, "test_xnli.tsv")
with open(filepath, 'w', encoding='utf-8') as f:
f.write("The cat is on the mat\tThe cat is on furniture\tentailment\n")
f.write("The dog is running\tThe cat is sleeping\tneutral\n")
f.write("It is raining outside\tIt is sunny outside\tcontradiction\n")
ARGS = {
'filepath': filepath
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| cl_forward |
Not yet run unverified (harvested, no recorded run) |
chongli17/crosslingualalignment/src/decoder_models.py code served (permissive licence) · get_code("add1c9f6ecb908c4") How it ran |