| SFL_MTSC |
Ran ran; our contract draft was wrong, not the code |
boyan1001/SFL-MTSC/self-consistency.py code served (permissive licence) · get_code("1248b9954a5da626") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import math
from collections import defaultdict, deque
from typing import Any, Dict, List, Set, Tuple
# Create sample frame data with domain, intent, and slots
frame1_path0 = {
'domain': 'restaurant',
'intent': 'book',
'slots': {'cuisine': 'italian', 'party_size': '4'}
}
frame2_path0 = {
'domain': 'restaurant',
'intent': 'book',
'slots': {'cuisine': 'italian', 'party_size': '4'}
}
frame3_path0 = {
'domain': 'restaurant',
'intent': 'book',
'slots': {'cuisine': 'french', 'party_size': '2'}
}
frame1_path1 = {
'domain': 'restaurant',
'intent': 'book',
'slots': {'cuisine': 'italian', 'party_size': '4'}
}
frame2_path1 = {
'domain': 'hotel',
'intent': 'search',
'slots': {'location': 'paris', 'stars': '5'}
}
frame3_path1 = {
'domain': 'hotel',
'intent': 'search',
'slots': {'location': 'paris', 'stars': '4'}
}
# Create semantics list - list of lists of frames
semantics = [
[frame1_path0, frame2_path0, frame3_path0],
[frame1_path1, frame2_path1, frame3_path1]
]
ARGS = {
'semantics': semantics,
'alpha': 0.3,
'tau': 0.55
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| cluster_hybrid_jaccard |
Ran ran; our contract draft was wrong, not the code |
boyan1001/SFL-MTSC/self-consistency.py code served (permissive licence) · get_code("0042b24f4b3203f4") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
from collections import defaultdict, deque
from typing import Any, Dict, List, Set, Tuple
# Build minimal frame dictionaries that the function expects
# The function calls frame.get('slots', {}) and expects a dict of key-value pairs
frame1 = {
'slots': {
'name': 'Alice',
'age': '30'
}
}
frame2 = {
'slots': {
'name': 'Alice',
'age': '30'
}
}
frame3 = {
'slots': {
'name': 'Bob',
'age': '25'
}
}
frame4 = {
'slots': {
'city': 'New York',
'state': 'NY'
}
}
# Create a list of frames to cluster
frames = [frame1, frame2, frame3, frame4]
# Define arguments for cluster_hybrid_jaccard
ARGS = {
'frames': frames,
'alpha': 0.3,
'tau': 0.55
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list — a list output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| compute_support |
Ran ran, honoured the contract we drafted |
boyan1001/SFL-MTSC/self-consistency.py code served (permissive licence) · get_code("92c01831e7a2db56") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned int, ndim 0, dtype family int, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
# The function expects a List[Dict] where each dict can contain a '_path_idx' key
# It iterates through the cluster, extracts '_path_idx' values, and counts unique indices
# Create a sample cluster with frames (dicts) containing '_path_idx' keys
cluster = [
{'_path_idx': 0, 'data': 'frame1'},
{'_path_idx': 1, 'data': 'frame2'},
{'_path_idx': 0, 'data': 'frame3'}, # duplicate path_idx
{'_path_idx': 2, 'data': 'frame4'},
{'data': 'frame5'}, # no '_path_idx' key
]
ARGS = {
'cluster': cluster
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type int, ndim 0, dtype family int, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| filter_slot_clusters |
Ran ran; our contract draft was wrong, not the code |
boyan1001/SFL-MTSC/self-consistency.py code served (permissive licence) · get_code("cf78332e15a4a1e4") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned defaultdict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import math
from collections import defaultdict, deque
from typing import Any, Dict, List, Set, Tuple
def compute_support(cluster: List[Dict]) -> int:
path_indices = set()
for frame in cluster:
p = frame.get('_path_idx')
if p is not None:
path_indices.add(p)
return len(path_indices)
def filter_slot_clusters(slot_clusters, k):
sup_threshold = math.ceil(k / 2)
retained = defaultdict(list)
for di_key, clusters in slot_clusters.items():
for cluster in clusters:
support = compute_support(cluster)
if support >= sup_threshold:
retained[di_key].append(cluster)
return retained
# Construct test data
slot_clusters = {
"slot_1": [
[
{"_path_idx": 0, "data": "frame0"},
{"_path_idx": 1, "data": "frame1"},
{"_path_idx": 0, "data": "frame2"},
],
[
{"_path_idx": 2, "data": "frame3"},
],
],
"slot_2": [
[
{"_path_idx": 0, "data": "frame4"},
{"_path_idx": 1, "data": "frame5"},
{"_path_idx": 2, "data": "frame6"},
],
[
{"_path_idx": 3, "data": "frame7"},
{"_path_idx": 4, "data": "frame8"},
],
],
}
k = 4
ARGS = {
"slot_clusters": slot_clusters,
"k": k,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type defaultdict — a defaultdict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a defaultdict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| hybrid_jaccard |
Ran ran, honoured the contract we drafted |
boyan1001/SFL-MTSC/self-consistency.py code served (permissive licence) · get_code("3dcb645e4d8116d4") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
from typing import Any, Dict, List, Set, Tuple
# Construct frame objects (dictionaries with 'slots' key)
frame_a = {
'slots': {
'restaurant_type': 'italian',
'location': 'downtown',
'price_range': 'moderate'
}
}
frame_b = {
'slots': {
'restaurant_type': 'italian',
'location': 'uptown',
'cuisine': 'pasta'
}
}
# Define ARGS with all parameters for hybrid_jaccard
ARGS = {
'fa': frame_a,
'fb': frame_b,
'alpha': 0.3
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| jaccard |
Ran ran, honoured the contract we drafted |
boyan1001/SFL-MTSC/self-consistency.py code served (permissive licence) · get_code("ffd9acbb3ca646ac") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
from typing import Any, Dict, List, Set, Tuple
# Test case 1: Two non-empty sets with some intersection
set_a = {1, 2, 3, 4, 5}
set_b = {3, 4, 5, 6, 7}
ARGS = {
"a": set_a,
"b": set_b,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| reintegrate_cluster |
Ran ran; our contract draft was wrong, not the code |
boyan1001/SFL-MTSC/self-consistency.py code served (permissive licence) · get_code("daf246cda5e400ef") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import math
from collections import defaultdict, deque
from typing import Any, Dict, List, Set, Tuple
# Construct valid arguments for reintegrate_cluster
# di_key: Tuple of (domain, intent)
di_key = ("restaurant", "book_table")
# cluster: List[Dict] - list of frame dictionaries with 'slots' key
cluster = [
{
'slots': {
'restaurant_type': 'italian',
'party_size': '4',
'time': '19:00'
}
},
{
'slots': {
'restaurant_type': 'italian',
'party_size': '4',
'location': 'downtown'
}
},
{
'slots': {
'restaurant_type': 'italian',
'party_size': '5',
'time': '19:00'
}
}
]
ARGS = {
'di_key': di_key,
'cluster': cluster
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| slot_pair |
Ran ran; our contract draft was wrong, not the code |
boyan1001/SFL-MTSC/self-consistency.py code served (permissive licence) · get_code("559ffc35906cec9b") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned set. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# Analyzing the function:
# - Takes 'frame' parameter
# - Calls frame.get('slots', {}) - so frame must be dict-like
# - Accesses slots as a dict with .items()
# - Returns a set of tuples of strings
# Create a simple frame dict with slots
frame = {
'slots': {
'slot_name_1': 'slot_value_1',
'slot_name_2': 'slot_value_2',
'slot_name_3': 42, # Test non-string value (will be converted to str)
}
}
ARGS = {
'frame': frame
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type set — a set output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a set output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| slot_values |
Ran ran; our contract draft was wrong, not the code |
boyan1001/SFL-MTSC/self-consistency.py code served (permissive licence) · get_code("2c009fecb1b66ecb") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned set. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from typing import Any, Dict, List, Set, Tuple
# Test case 1: frame with slots as a dict
frame_with_slots = {
'slots': {
'name': 'John',
'age': 25,
'city': 'New York'
}
}
# Test case 2: frame with empty slots dict
frame_empty_slots = {
'slots': {}
}
# Test case 3: frame with slots as None
frame_none_slots = {
'slots': None
}
# Test case 4: frame with slots as non-dict (invalid)
frame_invalid_slots = {
'slots': 'not a dict'
}
# Test case 5: frame without slots key
frame_no_slots_key = {
'other_key': 'value'
}
# Use the first test case as the main argument
ARGS = {
'frame': frame_with_slots
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type set — a set output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a set output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| voting_key |
Ran ran; our contract draft was wrong, not the code |
boyan1001/SFL-MTSC/self-consistency.py code served (permissive licence) · get_code("b951e0aff025c2ea") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import math
from collections import defaultdict, deque
from typing import Any, Dict, List, Set, Tuple
# Construct valid arguments for voting_key function
ARGS = {
"cluster": [
{
"slots": {
"entity_type": "person",
"name": "Alice",
"confidence": "0.95"
}
},
{
"slots": {
"entity_type": "person",
"name": "Alice",
"confidence": "0.92"
}
},
{
"slots": {
"entity_type": "organization",
"name": "Bob",
"confidence": "0.85"
}
}
]
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|