| block_mean_residual |
Ran ran; our contract draft was wrong, not the code |
skydancerosel/coactivation-closure/pipeline/layer_residualized_clustering.py code served (permissive licence) · get_code("0a27e5f8df47991a") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 2, dtype family float, shape (6, 6) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Construct arguments for block_mean_residual
n_layer = 2
n_head = 3
F = n_layer * n_head # 6
# Create a random positive matrix of shape (F, F) = (6, 6)
# This represents absolute Jacobian values
absJ = np.random.rand(F, F) * 10.0 # Scale to make it more interesting
ARGS = {
"absJ": absJ,
"n_layer": n_layer,
"n_head": n_head,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ndarray, ndim 2, dtype family float, shape (6, 6) — shape (6, 6) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| classify_all_heads |
Ran ran; our contract draft was wrong, not the code |
skydancerosel/coactivation-closure/analysis/supervised_vs_unsupervised.py code served (permissive licence) · get_code("0c78886f94b3f6eb") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# Construct all_head_selectivity dict
# Format: keys are "l{layer}_h{head}", values are dicts with class names and selectivity scores
all_head_selectivity = {
'l0_h0': {
'baseline': 0.5,
'class_a': 35.2,
'class_b': 15.1
},
'l0_h1': {
'baseline': 0.3,
'class_x': 28.5
},
'l1_h0': {
'baseline': 0.6,
'class_p': 32.0,
'class_q': 18.9
},
'l1_h2': {
'baseline': 0.4,
'class_m': 25.0
}
}
ARGS = {
'all_head_selectivity': all_head_selectivity,
'threshold': 30.0
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| cluster_and_score |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
skydancerosel/coactivation-closure/pipeline/layer_residualized_clustering.py code served (permissive licence) · get_code("8d7c9bc62f2becdd") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Construct arguments for cluster_and_score
# A: affinity matrix (precomputed similarity), shape (n_samples, n_samples)
# n_clusters: number of clusters
# gt_map: dictionary mapping (layer, head) tuples to ground truth class labels
# n_layer: number of layers
# n_head: number of heads per layer
n_layer = 2
n_head = 3
F = n_layer * n_head # Total number of features/samples = 6
# Create a small affinity matrix (6x6 symmetric matrix)
# Add some structure so clustering is meaningful
A = np.array([
[1.0, 0.8, 0.7, 0.2, 0.1, 0.15],
[0.8, 1.0, 0.75, 0.25, 0.15, 0.1],
[0.7, 0.75, 1.0, 0.3, 0.2, 0.15],
[0.2, 0.25, 0.3, 1.0, 0.85, 0.8],
[0.1, 0.15, 0.2, 0.85, 1.0, 0.9],
[0.15, 0.1, 0.15, 0.8, 0.9, 1.0]
], dtype=np.float32)
n_clusters = 2
# Ground truth map: (layer, head) -> class label
# Map some of the layer-head pairs to ground truth classes
gt_map = {
(0, 0): 'classA',
(0, 1): 'classA',
(0, 2): 'classA',
(1, 0): 'classB',
(1, 1): 'classB',
}
ARGS = {
'A': A,
'n_clusters': n_clusters,
'gt_map': gt_map,
'n_layer': n_layer,
'n_head': n_head,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, sklearn — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| confusion_matrix |
Ran ran; our contract draft was wrong, not the code |
skydancerosel/coactivation-closure/analysis/supervised_vs_unsupervised.py code served (permissive licence) · get_code("8c51cbf46667312d") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Construct valid arguments for confusion_matrix
# Parameters:
# - class_map: dict (L, H) -> class name
# - labels: array of length F = n_layer * n_head, cluster assignment
# - n_layer: number of layers
# - n_head: number of heads
n_layer = 3
n_head = 4
F = n_layer * n_head # Total number of features = 12
# Build class_map: dict mapping (L, H) tuples to class names
# We need to cover some (L, H) pairs with specialized classes and some with 'unspecialized'
class_map = {
(0, 0): 'class_A',
(0, 1): 'class_B',
(0, 2): 'class_A',
(1, 0): 'class_C',
(1, 1): 'unspecialized',
(1, 2): 'class_B',
(2, 0): 'class_A',
(2, 1): 'class_C',
# Some entries left out to test the .get() with default 'unspecialized'
}
# Build labels: array of length F with cluster assignments
# Use various cluster IDs (0, 1, 2, etc.)
labels = np.array([0, 1, 0, 2, 1, 1, 0, 2, 1, 0, 2, 1])
ARGS = {
'class_map': class_map,
'labels': labels,
'n_layer': n_layer,
'n_head': n_head,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple — a tuple output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| evaluate |
Ran ran; our contract draft was wrong, not the code |
skydancerosel/coactivation-closure/closure/closure_olmo_natural.py code served (permissive licence) · get_code("1e52205f22b814fa") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
import numpy as np
torch.manual_seed(0)
# Create a minimal model that has a .logits attribute and accepts tokens
class MinimalModel(nn.Module):
def __init__(self, vocab_size=10, seq_len=5, hidden_dim=4):
super().__init__()
self.embedding = nn.Embedding(vocab_size, hidden_dim)
self.linear = nn.Linear(hidden_dim, vocab_size)
def forward(self, tokens):
# tokens shape: (batch_size, seq_len)
x = self.embedding(tokens) # (batch_size, seq_len, hidden_dim)
logits = self.linear(x) # (batch_size, seq_len, vocab_size)
# Return object with .logits attribute
class Output:
pass
out = Output()
out.logits = logits
return out
model = MinimalModel(vocab_size=10, seq_len=5, hidden_dim=4)
# tokens: shape (n_samples, seq_len) - token ids
n_samples = 2
seq_len = 5
tokens = torch.randint(0, 10, (n_samples, seq_len), dtype=torch.long)
# query_pos: shape (n_samples,) - position index in sequence for each sample
query_pos = torch.tensor([1, 2], dtype=torch.long)
# targets: shape (n_samples,) - target token id for each sample
targets = torch.tensor([3, 5], dtype=torch.long)
# device
device = 'cpu'
# batch_size
batch_size = 2
ARGS = {
'model': model,
'tokens': tokens,
'query_pos': query_pos,
'targets': targets,
'device': device,
'batch_size': batch_size
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 2, dtype family float — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| evaluate |
Ran ran; our contract draft was wrong, not the code |
skydancerosel/coactivation-closure/closure/closure_olmoe_route_cluster1.py code served (permissive licence) · get_code("acf74f6ad99f9f97") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
import numpy as np
torch.manual_seed(0)
class DummyModel(nn.Module):
def __init__(self, vocab_size=10, hidden_dim=4):
super().__init__()
self.hidden_dim = hidden_dim
self.vocab_size = vocab_size
self.linear = nn.Linear(hidden_dim, vocab_size)
def forward(self, tokens):
# tokens shape: (batch_size, seq_len)
# Return object with .logits attribute
batch_size, seq_len = tokens.shape
# Create dummy hidden states
hidden = torch.randn(batch_size, seq_len, self.hidden_dim)
logits = self.linear(hidden)
class LogitsOutput:
pass
output = LogitsOutput()
output.logits = logits
return output
# Create model
model = DummyModel(vocab_size=10, hidden_dim=4)
# Create tokens: (batch_size=2, seq_len=5)
tokens = torch.randint(0, 10, (2, 5), dtype=torch.long)
# Create query_pos: positions in sequence where we evaluate (shape: 2)
query_pos = torch.tensor([2, 3], dtype=torch.long)
# Create targets: target token ids at those positions (shape: 2)
targets = torch.tensor([5, 7], dtype=torch.long)
# Device
device = 'cpu'
# Batch size
batch_size = 2
ARGS = {
'model': model,
'tokens': tokens,
'query_pos': query_pos,
'targets': targets,
'device': device,
'batch_size': batch_size
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 2, dtype family float — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| evaluate |
Ran ran; our contract draft was wrong, not the code |
skydancerosel/coactivation-closure/closure/closure_test_olmo_cluster2.py code served (permissive licence) · get_code("8fa9b2fb8d43738a") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
import numpy as np
torch.manual_seed(0)
class SimpleModel(nn.Module):
def __init__(self, vocab_size=10, hidden_dim=4):
super().__init__()
self.embedding = nn.Embedding(vocab_size, hidden_dim)
self.linear = nn.Linear(hidden_dim, vocab_size)
def forward(self, x):
x = self.embedding(x)
logits = self.linear(x)
return type('Output', (), {'logits': logits})()
model = SimpleModel(vocab_size=10, hidden_dim=4)
model.eval()
# tokens: shape (batch_size, seq_len) with token indices
tokens = torch.randint(0, 10, (2, 5), dtype=torch.long)
# targets: shape (batch_size,) with class indices
targets = torch.randint(0, 10, (2,), dtype=torch.long)
device = 'cpu'
batch_size = 8
ARGS = {
'model': model,
'tokens': tokens,
'targets': targets,
'device': device,
'batch_size': batch_size
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| evaluate |
Ran ran; our contract draft was wrong, not the code |
skydancerosel/coactivation-closure/developmental/closure_at_intermediate.py code served (permissive licence) · get_code("97a35722da7538dd") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
import numpy as np
torch.manual_seed(0)
class SimpleModel(nn.Module):
def __init__(self, vocab_size=10, hidden_dim=4):
super().__init__()
self.embedding = nn.Embedding(vocab_size, hidden_dim)
self.linear = nn.Linear(hidden_dim, vocab_size)
def forward(self, tokens):
x = self.embedding(tokens)
logits = self.linear(x)
# Return object with .logits attribute
class Output:
pass
out = Output()
out.logits = logits
return out
# Create a small model
model = SimpleModel(vocab_size=10, hidden_dim=4)
# Create tiny tensors
# tokens: shape (2, 3) - 2 sequences of length 3, values in [0, 9]
tokens = torch.randint(0, 10, (2, 3), dtype=torch.long)
# query_pos: shape (2,) - position indices for each sequence, values in [0, 2]
query_pos = torch.tensor([0, 1], dtype=torch.long)
# targets: shape (2,) - target token indices, values in [0, 9]
targets = torch.tensor([3, 7], dtype=torch.long)
# device: cpu only
device = 'cpu'
ARGS = {
'model': model,
'tokens': tokens,
'query_pos': query_pos,
'targets': targets,
'device': device,
'batch_size': 2
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 2, dtype family float — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| evaluate_loss |
Ran ran; our contract draft was wrong, not the code |
skydancerosel/coactivation-closure/developmental/olmoe_baseline_loss_scan.py code served (permissive licence) · get_code("3b2c30ae4c8e6d25") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
import numpy as np
torch.manual_seed(0)
# Create a minimal model that has a forward method returning an object with .logits
class MinimalModel(nn.Module):
def __init__(self):
super().__init__()
self.linear = nn.Linear(4, 8) # input dim 4, output vocab size 8
def forward(self, x):
# x shape: (batch, seq_len) with token ids
# Need to embed and produce logits
batch_size, seq_len = x.shape
logits = self.linear(torch.randn(batch_size, seq_len, 4)) # (batch, seq_len, vocab_size=8)
class Output:
pass
output = Output()
output.logits = logits
return output
model = MinimalModel()
# Create small tensors
# tokens: shape (n_samples, seq_len) with token indices
tokens = torch.randint(0, 8, (2, 3))
# query_pos: shape (n_samples,) with position indices in sequence
query_pos = torch.tensor([0, 1])
# targets: shape (n_samples,) with target token indices
targets = torch.tensor([3, 5])
device = 'cpu'
ARGS = {
'model': model,
'tokens': tokens,
'query_pos': query_pos,
'targets': targets,
'device': device,
'batch_size': 4
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| fit_ising |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
skydancerosel/coactivation-closure/closure/select_closure_candidate.py code served (permissive licence) · get_code("a4e1334eaf63a0e0") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 2, dtype family float, shape (4, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Create a small spins array with shape (n_samples, n_features)
# Using distinct dimensions: 5 samples, 4 features
spins = np.array([
[1.0, -1.0, 1.0, -1.0],
[-1.0, 1.0, -1.0, 1.0],
[1.0, 1.0, -1.0, -1.0],
[-1.0, -1.0, 1.0, 1.0],
[1.0, -1.0, 1.0, 1.0]
], dtype=np.float64)
ARGS = {
"spins": spins,
"l2": 0.001
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, sklearn — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ndarray, ndim 2, dtype family float, shape (4, 4) — shape (4, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| install_ablation |
Ran ran; our contract draft was wrong, not the code |
skydancerosel/coactivation-closure/closure/closure_olmo_natural.py code served (permissive licence) · get_code("8fd8b2b800c739f4") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Build a minimal model structure that matches what install_ablation expects
class SelfAttention(nn.Module):
def __init__(self):
super().__init__()
# o_proj needs to be a module that can have register_forward_pre_hook called on it
self.o_proj = nn.Linear(4, 4)
def forward(self, x):
return self.o_proj(x)
class TransformerLayer(nn.Module):
def __init__(self):
super().__init__()
self.self_attn = SelfAttention()
def forward(self, x):
return self.self_attn(x)
class SimpleModel(nn.Module):
def __init__(self, num_layers=3):
super().__init__()
self.model = nn.Module()
self.model.layers = nn.ModuleList([TransformerLayer() for _ in range(num_layers)])
def forward(self, x):
for layer in self.model.layers:
x = layer(x)
return x
# Create model
model = SimpleModel(num_layers=3)
# ablate_dict: maps layer_idx to list of head indices to ablate
# Using layer indices 0, 1, 2 and some head indices
ablate_dict = {
0: [0, 1], # Ablate heads 0, 1 in layer 0
1: [2], # Ablate head 2 in layer 1
2: [], # No ablation in layer 2
}
# head_dim: dimension of each attention head
head_dim = 4
ARGS = {
"model": model,
"ablate_dict": ablate_dict,
"head_dim": head_dim,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| install_ablation |
Ran ran; our contract draft was wrong, not the code |
skydancerosel/coactivation-closure/closure/closure_olmoe_route_cluster1.py code served (permissive licence) · get_code("465531a9283db4d0") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Build a minimal model structure that matches what install_ablation expects
class MinimalAttention(nn.Module):
def __init__(self):
super().__init__()
self.o_proj = nn.Linear(4, 4)
def forward(self, x):
return self.o_proj(x)
class MinimalLayer(nn.Module):
def __init__(self):
super().__init__()
self.self_attn = MinimalAttention()
def forward(self, x):
return self.self_attn(x)
class MinimalModel(nn.Module):
def __init__(self, num_layers=3):
super().__init__()
self.layers = nn.ModuleList([MinimalLayer() for _ in range(num_layers)])
def forward(self, x):
for layer in self.layers:
x = layer(x)
return x
class MinimalWrapper(nn.Module):
def __init__(self):
super().__init__()
self.model = MinimalModel(num_layers=3)
def forward(self, x):
return self.model(x)
# Create instances for arguments
model = MinimalWrapper()
# ablate_dict: dict mapping layer_idx to list of head indices to zero
# Using layer indices 0 and 2, with different head indices
ablate_dict = {
0: [0, 1], # ablate heads 0 and 1 in layer 0
1: [], # no ablation in layer 1
2: [2], # ablate head 2 in layer 2
}
# head_dim: dimension of each attention head
head_dim = 2
ARGS = {
"model": model,
"ablate_dict": ablate_dict,
"head_dim": head_dim,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| install_ablation |
Ran ran; our contract draft was wrong, not the code |
skydancerosel/coactivation-closure/closure/closure_pythia_natural.py code served (permissive licence) · get_code("9b4fd51b124e8e38") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
class DenseModule(nn.Module):
def __init__(self, head_dim):
super().__init__()
self.linear = nn.Linear(head_dim * 8, head_dim * 8)
def forward(self, x):
return self.linear(x)
class AttentionModule(nn.Module):
def __init__(self, head_dim):
super().__init__()
self.dense = DenseModule(head_dim)
class LayerModule(nn.Module):
def __init__(self, head_dim):
super().__init__()
self.attention = AttentionModule(head_dim)
class GPTNeoXModule(nn.Module):
def __init__(self, head_dim, num_layers):
super().__init__()
self.layers = nn.ModuleList([LayerModule(head_dim) for _ in range(num_layers)])
class GPTModel(nn.Module):
def __init__(self, head_dim, num_layers):
super().__init__()
self.gpt_neox = GPTNeoXModule(head_dim, num_layers)
head_dim = 4
num_layers = 3
model = GPTModel(head_dim, num_layers)
ablate_dict = {
0: [0, 1],
1: [2],
2: []
}
ARGS = {
"model": model,
"ablate_dict": ablate_dict,
"head_dim": head_dim
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| install_ablation |
Ran ran; our contract draft was wrong, not the code |
skydancerosel/coactivation-closure/developmental/closure_at_intermediate.py code served (permissive licence) · get_code("42a0a461d8483ebe") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Build minimal models for both Pythia and non-Pythia architectures
# We'll create a Pythia-style model since the function checks for it
class DenseModule(nn.Module):
def __init__(self, in_dim, out_dim):
super().__init__()
self.linear = nn.Linear(in_dim, out_dim)
def forward(self, x):
return self.linear(x)
class AttentionModule(nn.Module):
def __init__(self, head_dim, num_heads):
super().__init__()
self.dense = DenseModule(head_dim * num_heads, head_dim * num_heads)
def forward(self, x):
return self.dense(x)
class LayerModule(nn.Module):
def __init__(self, head_dim, num_heads):
super().__init__()
self.attention = AttentionModule(head_dim, num_heads)
def forward(self, x):
return self.attention(x)
class PythiaModel(nn.Module):
def __init__(self, num_layers, head_dim, num_heads):
super().__init__()
self.gpt_neox = nn.Module()
self.gpt_neox.layers = nn.ModuleList([
LayerModule(head_dim, num_heads) for _ in range(num_layers)
])
def forward(self, x):
return x
class LlamaLikeModel(nn.Module):
def __init__(self, num_layers, head_dim, num_heads):
super().__init__()
self.model = nn.Module()
class SelfAttn(nn.Module):
def __init__(self, hidden_dim):
super().__init__()
self.o_proj = DenseModule(hidden_dim, hidden_dim)
def forward(self, x):
return self.o_proj(x)
self.model.layers = nn.ModuleList([
nn.Module() for _ in range(num_layers)
])
for i in range(num_layers):
self.model.layers[i].self_attn = SelfAttn(head_dim * num_heads)
def forward(self, x):
return x
# Create test model (Pythia style)
head_dim = 4
num_heads = 3
num_layers = 2
model = PythiaModel(num_layers, head_dim, num_heads)
model_name = "pythia-70m"
# Define ablation dictionary: layer_idx -> list of head indices to ablate
ablate_dict = {
0: [0, 1], # Ablate heads 0 and 1 in layer 0
1: [2], # Ablate head 2 in layer 1
}
ARGS = {
"model": model,
"model_name": model_name,
"ablate_dict": ablate_dict,
"head_dim": head_dim,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| make_pre_hook |
Ran ran; our contract draft was wrong, not the code |
skydancerosel/coactivation-closure/closure/closure_olmo_natural.py code served (permissive licence) · get_code("7d0df836b41c41f0") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned function. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# heads_in_layer: a list of head indices to process
heads_in_layer = [0, 2]
# head_dim: the dimension size of each head
head_dim = 5
ARGS = {
"heads_in_layer": heads_in_layer,
"head_dim": head_dim,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type function — a function output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a function output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| per_class_recall_precision |
Ran ran; our contract draft was wrong, not the code |
skydancerosel/coactivation-closure/analysis/supervised_vs_unsupervised.py code served (permissive licence) · get_code("5473f133c9597574") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Construct a confusion matrix (4 classes x 3 clusters)
cm = np.array([
[10, 2, 1], # class 0: mostly in cluster 0
[1, 15, 2], # class 1: mostly in cluster 1
[3, 1, 8], # class 2: mostly in cluster 2
[0, 0, 0], # class 3: empty class
], dtype=np.float64)
# List of class names/identifiers
classes = ['cat', 'dog', 'bird', 'fish']
# List of cluster identifiers
clusters = [10, 20, 30]
ARGS = {
'cm': cm,
'classes': classes,
'clusters': clusters,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| per_head_median_split |
Ran ran, honoured the contract we drafted |
skydancerosel/coactivation-closure/closure/select_closure_candidate.py code served (permissive licence) · get_code("caa2d8429b3ae290") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Create a small signal array with shape (batch_size, num_heads)
# The function computes median along axis 0 (across batch), then compares element-wise
signal = np.array([
[1.5, 2.0, 0.5, 3.2],
[0.8, 1.5, 1.2, 2.1],
], dtype=np.float32)
ARGS = {
"signal": signal
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ndarray, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| within_layer_cluster |
Ran ran; our contract draft was wrong, not the code |
skydancerosel/coactivation-closure/pipeline/layer_residualized_clustering.py code served (permissive licence) · get_code("805a6e705b42ec63") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family int, shape (12,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Construct arguments for within_layer_cluster
n_layer = 3
n_head = 4
k_per_layer = 2
# Create a symmetric Jacobian-like matrix (required for SpectralClustering with affinity='precomputed')
# Shape: (n_layer * n_head, n_layer * n_head)
total_heads = n_layer * n_head
J = np.random.randn(total_heads, total_heads)
# Make it symmetric and positive (SpectralClustering with precomputed affinity needs similarity matrix)
J = (J + J.T) / 2
J = np.abs(J) + 0.1 # Ensure positive values for affinity matrix
ARGS = {
"J": J,
"n_layer": n_layer,
"n_head": n_head,
"k_per_layer": k_per_layer
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, sklearn — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ndarray, ndim 1, dtype family int, shape (12,) — shape (12,) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| build_induction_batch |
Not yet run unverified (harvested, no recorded run) |
skydancerosel/coactivation-closure/closure/closure_test_olmo_cluster2.py code served (permissive licence) · get_code("7cf4e93599c5a110") How it ran |
| template_free_signal |
Not yet run unverified (harvested, no recorded run) |
skydancerosel/coactivation-closure/closure/select_closure_candidate.py code served (permissive licence) · get_code("7a80a252257f4ac4") How it ran |