| compute_measure |
Ran ran on a synthesized input |
openai/grok/grok/metrics.py code served (permissive licence) · get_code("8e9ee805c0204396") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
import math
torch.manual_seed(0)
class SimpleModel(nn.Module):
def __init__(self):
super().__init__()
self.linear1 = nn.Linear(4, 3)
self.linear2 = nn.Linear(3, 2)
def forward(self, x):
x = self.linear1(x)
x = self.linear2(x)
return x
def simple_measure_func(module, init_module, **kwargs):
"""
Simple measure function that computes the L2 norm of weight difference.
"""
if isinstance(module, nn.Linear):
weight_diff = (module.weight - init_module.weight).norm()
return weight_diff.item() + 1e-8 # Add small epsilon to avoid log(0)
return 1e-8
# Create model and init_model
model = SimpleModel()
init_model = SimpleModel()
# Store initial weights
for m, im in zip(model.modules(), init_model.modules()):
if isinstance(m, nn.Linear):
im.weight.data = m.weight.data.clone()
im.bias.data = m.bias.data.clone()
# Simulate training by modifying weights slightly
for param in model.parameters():
param.data += torch.randn_like(param) * 0.1
# Define ARGS
ARGS = {
'model': model,
'init_model': init_model,
'measure_func': simple_measure_func,
'operator': 'sum',
'kwargs': {},
'p': 2
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| factor_expts |
Ran ran on a synthesized input |
openai/grok/grok/visualization.py code served (permissive licence) · get_code("e95fa4ee731c03ea") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# The function factor_expts takes a list of experiment name strings
# It parses each string by splitting on '_', extracts architecture and time parameters
# The format appears to be: word_word_word_word-t_...
# where the first 3 words form the architecture, and the 4th word contains '-t' pattern
expts = [
"resnet_layer_basic_t-5_extra",
"resnet_layer_basic_t-10_extra",
"vgg_conv_small_t-3_config",
"vgg_conv_small_t-7_config",
"bert_attention_head_t-2_setting",
]
ARGS = {
"expts": expts
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_device |
Ran ran on a synthesized input |
danielmamay/grokking/grokking/training.py code served (permissive licence) · get_code("c1c618e180dfc666") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned device. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# The function get_device takes a single parameter 'device' which is a string
# It handles 'auto', 'cuda', 'mps', and 'cpu' device specifications
# We'll test with 'cpu' since we're on CPU-only environment
ARGS = {
'device': 'cpu'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type device — a device output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a device output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_loss_and_grads |
Ran ran on a synthesized input |
openai/grok/grok/measure.py code served (permissive licence) · get_code("61f1915864fc4345") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
import numpy as np
torch.manual_seed(0)
class SimpleModel(nn.Module):
def __init__(self):
super().__init__()
self.linear = nn.Linear(4, 3)
def _step(self, batch, batch_idx, train, grads):
x, y = batch
output = self.linear(x)
loss = torch.mean((output - y) ** 2)
if grads:
loss.backward()
grad_list = []
for p in self.parameters():
if p.grad is not None:
grad_list.append(p.grad.clone().flatten())
else:
grad_list.append(torch.zeros_like(p).flatten())
all_grads = torch.cat(grad_list)
return loss.detach(), all_grads
else:
return loss.detach(), None
model = SimpleModel()
# Initialize x with flattened parameters
x_list = []
for p in model.parameters():
x_list.append(p.data.cpu().numpy().flatten())
x = np.concatenate(x_list).astype(np.float64)
# Create data loader with small batches
data_loader = [
(torch.randn(2, 4), torch.randn(2, 3)),
(torch.randn(2, 4), torch.randn(2, 3))
]
ARGS = {
'x': x,
'model': model,
'data_loader': data_loader
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple — a tuple output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_sharpness |
Ran ran on a synthesized input |
openai/grok/grok/measure.py code served (permissive licence) · get_code("739eddc60be1c197") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
import numpy as np
torch.manual_seed(0)
class SimpleModel(nn.Module):
def __init__(self):
super().__init__()
self.linear1 = nn.Linear(4, 3)
self.linear2 = nn.Linear(3, 2)
def forward(self, x):
x = torch.relu(self.linear1(x))
x = self.linear2(x)
return x
def _step(self, batch, batch_idx, train, grads):
x, y = batch
logits = self.forward(x)
loss = nn.functional.mse_loss(logits, y)
if grads:
loss.backward()
grad_list = []
for p in self.parameters():
if p.grad is not None:
grad_list.append(p.grad.data.view(-1))
else:
grad_list.append(torch.zeros_like(p.data.view(-1)))
grads_tensor = torch.cat(grad_list)
return loss.detach(), grads_tensor.detach()
else:
return loss.detach(), None
model = SimpleModel()
data_loader = [
(torch.randn(2, 4), torch.randn(2, 2)),
(torch.randn(2, 4), torch.randn(2, 2)),
]
ARGS = {
'data_loader': data_loader,
'model': model,
'subspace_dim': 5,
'epsilon': 0.01,
'maxiter': 2,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, scipy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| gradfilter_ema |
Ran ran; our contract draft was wrong, not the code |
ironjr/grokfast/grokfast.py code served (permissive licence) · get_code("b1849c24fbb69bfa") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Create a simple neural network module
class SimpleModel(nn.Module):
def __init__(self):
super().__init__()
self.linear1 = nn.Linear(4, 3)
self.linear2 = nn.Linear(3, 2)
def forward(self, x):
x = self.linear1(x)
x = self.linear2(x)
return x
# Instantiate the model
m = SimpleModel()
# Create a simple batch of data and compute gradients
x = torch.randn(2, 4)
target = torch.randint(0, 2, (2,))
# Forward pass
output = m(x)
loss = nn.CrossEntropyLoss()(output, target)
# Backward pass to populate gradients
loss.backward()
# Create initial grads dict (can be None, but let's test with a value too)
grads = {n: p.grad.data.detach().clone() for n, p in m.named_parameters() if p.requires_grad and p.grad is not None}
ARGS = {
"m": m,
"grads": grads,
"alpha": 0.98,
"lamb": 2.0
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| gradfilter_ma |
Ran ran; our contract draft was wrong, not the code |
ironjr/grokfast/grokfast.py code served (permissive licence) · get_code("2617e928ebfb8675") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from collections import deque
from typing import Dict, Optional, Literal
torch.manual_seed(0)
# Create a simple model with parameters that have gradients
class SimpleModel(nn.Module):
def __init__(self):
super().__init__()
self.linear1 = nn.Linear(4, 3)
self.linear2 = nn.Linear(3, 2)
def forward(self, x):
x = self.linear1(x)
x = self.linear2(x)
return x
# Build the model
m = SimpleModel()
# Create some dummy input and compute gradients
x = torch.randn(2, 4)
y = torch.randn(2, 2)
output = m(x)
loss = ((output - y) ** 2).sum()
loss.backward()
# Create grads dict with deques containing gradient tensors
window_size = 100
grads = {n: deque(maxlen=window_size) for n, p in m.named_parameters() if p.requires_grad and p.grad is not None}
# Populate with initial gradient values
for n, p in m.named_parameters():
if p.requires_grad and p.grad is not None:
grads[n].append(p.grad.data.detach().clone())
ARGS = {
'm': m,
'grads': grads,
'window_size': 100,
'lamb': 5.0,
'filter_type': 'mean',
'warmup': False,
'trigger': False
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| load_metric_data |
Ran ran on a synthesized input |
openai/grok/grok/visualization.py code served (permissive licence) · get_code("7564745b254db80c") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import os
import tempfile
import csv
from pathlib import Path
torch.manual_seed(0)
# Create a temporary directory structure with mock data
temp_dir = tempfile.mkdtemp()
# Create experiment subdirectories with metrics.csv files
expt_names = [
"arch_v1_base_t-10",
"arch_v1_base_t-20",
"arch_v2_wide_t-15",
]
for expt_name in expt_names:
expt_path = os.path.join(temp_dir, expt_name, "default", "version_0")
os.makedirs(expt_path, exist_ok=True)
# Create a minimal metrics.csv
csv_path = os.path.join(expt_path, "metrics.csv")
with open(csv_path, 'w', newline='') as f:
writer = csv.DictWriter(f, fieldnames=[
'learning_rate', 'train_loss', 'train_accuracy',
'val_loss', 'val_accuracy', 'epoch'
])
writer.writeheader()
# Write 100000 rows of dummy data to satisfy epochs=100000
for epoch in range(100000):
writer.writerow({
'learning_rate': '0.001',
'train_loss': '0.5',
'train_accuracy': '0.95',
'val_loss': '0.6',
'val_accuracy': '0.94',
'epoch': str(epoch)
})
ARGS = {
'data_dir': temp_dir,
'epochs': 100000,
'load_partial_data': False,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch, tqdm — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| most_interesting |
Ran ran on a synthesized input |
openai/grok/grok/visualization.py code served (permissive licence) · get_code("ecac6edaeb33872c") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# Construct metric_data dict with architecture keys
# Each architecture has 'T', 'val_accuracy', and 'val_loss' tensors
metric_data = {
'arch1': {
'T': torch.tensor([1, 2, 3, 4]),
'val_accuracy': torch.tensor([[92.0, 94.0, 96.0, 97.0],
[91.0, 93.0, 95.0, 98.0]]),
'val_loss': torch.tensor([[0.5, 0.3, 0.2, 0.1],
[0.6, 0.4, 0.25, 0.08]])
},
'arch2': {
'T': torch.tensor([1, 2, 3, 4]),
'val_accuracy': torch.tensor([[88.0, 90.0, 92.0, 94.0],
[89.0, 91.0, 93.0, 95.5]]),
'val_loss': torch.tensor([[0.8, 0.6, 0.4, 0.3],
[0.7, 0.5, 0.35, 0.2]])
}
}
ARGS = {
'metric_data': metric_data
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| norm |
Ran ran, honoured the contract we drafted |
openai/grok/grok/metrics.py code served (permissive licence) · get_code("458780ffac6c9c58") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
class SimpleModule(nn.Module):
def __init__(self):
super().__init__()
# Create a weight matrix with distinct dimensions (4 output units, 3 input features)
self.weight = nn.Parameter(torch.randn(4, 3))
module = SimpleModule()
init_module = SimpleModule()
ARGS = {
"module": module,
"init_module": init_module,
"p": 2,
"q": 2
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| op_norm |
Ran ran on a synthesized input |
openai/grok/grok/metrics.py code served (permissive licence) · get_code("7a0e73ebc4aab88f") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
class SimpleModule(nn.Module):
def __init__(self):
super().__init__()
self.weight = nn.Parameter(torch.randn(4, 3))
def forward(self, x):
return x @ self.weight.t()
module = SimpleModule()
init_module = SimpleModule()
p = float('Inf')
ARGS = {
'module': module,
'init_module': init_module,
'p': p
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_data_loaders |
Not yet run unverified (harvested, no recorded run) |
danielmamay/grokking/grokking/data.py code served (permissive licence) · get_code("ab942b3fd9d96975") How it ran |
| get_weights |
Not yet run unverified (harvested, no recorded run) |
openai/grok/grok/measure.py code served (permissive licence) · get_code("269ed6282c593f2f") How it ran |
| operation_mod_p_data |
Not yet run unverified (harvested, no recorded run) |
danielmamay/grokking/grokking/data.py code served (permissive licence) · get_code("f02ef440c5bcd946") How it ran |