| DiscreteOneHotTaskEncoder |
Ran ran on a synthesized input |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("21cbf833527e000d") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from torch import nn
class DiscreteOneHotTaskEncoder(nn.Module):
def __init__(self, task_emb_size: int, n_classes: int, n_embs: int):
super().__init__()
self.n_classes = n_classes
self.n_embs = n_embs
self.encoded_task_emb_size = n_classes * n_embs
self.mlp = nn.Sequential(nn.Linear(task_emb_size, self.encoded_task_emb_size * 4), nn.SiLU(), nn.Dropout(0.05), nn.Linear(self.encoded_task_emb_size * 4, self.encoded_task_emb_size))
def forward(self, x: torch.Tensor) -> dict[str, torch.Tensor]:
logits = self.mlp(x)
logits = logits.reshape(*logits.shape[:-1], self.n_embs, self.n_classes)
if self.training:
dist = torch.distributions.one_hot_categorical.OneHotCategorical(logits=logits)
one_hot = dist.sample() + dist.probs - dist.probs.detach()
else:
one_hot = torch.eye(self.n_classes, device=x.device)[torch.argmax(logits, dim=-1)]
return {'encoded_task_emb': torch.flatten(one_hot, start_dim=-2, end_dim=-1)}
INIT_ARGS = {
'task_emb_size': 4,
'n_classes': 3,
'n_embs': 2,
}
ARGS = {
'x': torch.randn(2, 4),
}
if __name__ == '__main__':
model = DiscreteOneHotTaskEncoder(**INIT_ARGS)
model.eval()
output = model(**ARGS)
print(output)
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| MLPResidualBlock |
Ran ran on a synthesized input |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("96865de164c308ad") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from torch import nn
class MLPResidualBlock(nn.Module):
def __init__(self, input_size, hidden_size, output_size, pre_layer_norm, post_dropout):
super().__init__()
layers = []
if pre_layer_norm:
layers.append(nn.LayerNorm(input_size))
layers += [nn.Linear(input_size, hidden_size), nn.SiLU(), nn.Dropout(0.05), nn.Linear(hidden_size, output_size), nn.SiLU()]
if post_dropout:
layers.append(nn.Dropout(0.05))
self.mlp = nn.Sequential(*layers)
def forward(self, x):
return x + self.mlp(x)
INIT_ARGS = {
'input_size': 4,
'hidden_size': 8,
'output_size': 4,
'pre_layer_norm': True,
'post_dropout': True,
}
ARGS = {
'x': torch.randn(2, 4),
}
if __name__ == '__main__':
block = MLPResidualBlock(**INIT_ARGS)
output = block(**ARGS)
print(f"Input shape: {ARGS['x'].shape}")
print(f"Output shape: {output.shape}")
print(f"Output:\n{output}")
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| SoftmaxTaskEncoder |
Ran ran on a synthesized input |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("aeca7914797f1f3a") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from torch import nn
import torch.nn.functional as F
class SoftmaxTaskEncoder(nn.Module):
def __init__(self, task_emb_size: int, code_dim: int, n_embs: int):
super().__init__()
self.n_embs = n_embs
self.code_dim = code_dim
self.mlp = nn.Sequential(nn.Linear(task_emb_size, code_dim * 4), nn.SiLU(), nn.Dropout(0.05), nn.Linear(code_dim * 4, n_embs))
self.embed = nn.Parameter(torch.randn(n_embs, code_dim), requires_grad=True)
self.ln = nn.LayerNorm(code_dim)
def forward(self, x: torch.Tensor) -> dict[str, torch.Tensor]:
logits = self.mlp(x).reshape(-1, self.n_embs)
probs = F.softmax(logits, dim=-1)
emb = probs.unsqueeze(-1) * self.embed.unsqueeze(0)
emb = emb.sum(1)
emb = self.ln(emb)
return {'encoded_task_emb': emb}
INIT_ARGS = {
'task_emb_size': 4,
'code_dim': 3,
'n_embs': 5,
}
ARGS = {
'x': torch.randn(2, 4),
}
if __name__ == '__main__':
model = SoftmaxTaskEncoder(**INIT_ARGS)
output = model(**ARGS)
print(output)
print(output['encoded_task_emb'].shape)
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| TaskEncoder |
Ran ran on a synthesized input |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("656b1d9383fc1abf") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from torch import nn
class TaskEncoder(nn.Module):
def __init__(self, task_emb_size: int, encoded_task_emb_size: int):
super().__init__()
self.encoded_task_emb_size = encoded_task_emb_size
self.mlp = nn.Sequential(nn.Linear(task_emb_size, encoded_task_emb_size), nn.LayerNorm(encoded_task_emb_size))
def get_one_hot_task_emb(self, num_tasks: int, task_idx: torch.Tensor) -> torch.Tensor:
return torch.eye(num_tasks, device=task_idx.device)[task_idx]
def forward(self, x: torch.Tensor) -> dict[str, torch.Tensor]:
return {'encoded_task_emb': self.mlp(x)}
INIT_ARGS = {
'task_emb_size': 4,
'encoded_task_emb_size': 3,
}
ARGS = {
'x': torch.randn(2, 4),
}
if __name__ == '__main__':
model = TaskEncoder(**INIT_ARGS)
output = model(**ARGS)
print(output)
print(output['encoded_task_emb'].shape)
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| VQTaskEncoder |
Ran ran on a synthesized input |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("2c3f7d0ccd979162") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from torch import nn
import torch.nn.functional as F
class VQTaskEncoder(nn.Module):
def __init__(self, task_emb_size: int, code_dim: int, n_embs: int, decay: float=0.99):
super().__init__()
self.n_embs = n_embs
self.code_dim = code_dim
self.decay = decay
self.mlp = nn.Sequential(nn.Linear(task_emb_size, code_dim * 4), nn.SiLU(), nn.Dropout(0.05), nn.Linear(code_dim * 4, code_dim))
self.ln = nn.LayerNorm(code_dim)
self.register_buffer('embed', torch.randn(code_dim, n_embs))
self.register_buffer('cluster_size', torch.zeros(n_embs))
self.register_buffer('embed_avg', self.embed.clone())
def embed_code(self, embed_id):
return F.embedding(embed_id, self.embed.transpose(0, 1))
def forward(self, x: torch.Tensor) -> dict[str, torch.Tensor]:
z_e = self.mlp(x).reshape(-1, self.code_dim)
dist = z_e.pow(2).sum(1, keepdim=True) - 2 * z_e @ self.embed + self.embed.pow(2).sum(0, keepdim=True)
indices = torch.argmin(dist, dim=-1)
one_hot = F.one_hot(indices, self.n_embs).to(z_e.dtype)
indices = indices.view(*x.shape[:-1])
quantize = self.embed_code(indices)
if self.training:
embed_onehot_sum = one_hot.sum(0)
embed_sum = z_e.transpose(0, 1) @ one_hot
self.cluster_size.data.mul_(self.decay).add_(embed_onehot_sum, alpha=1 - self.decay)
self.embed_avg.data.mul_(self.decay).add_(embed_sum, alpha=1 - self.decay)
n = self.cluster_size.sum()
cluster_size = (self.cluster_size + 1e-05) / (n + self.n_embs * 1e-05) * n
embed_normalized = self.embed_avg / cluster_size.unsqueeze(0)
self.embed.data.copy_(embed_normalized)
commitment_cost = 0.01
diff = commitment_cost * (quantize.detach() - z_e).pow(2).mean()
quantize = self.ln(z_e + (quantize - z_e).detach())
return {'encoded_task_emb': quantize, 'loss': diff}
INIT_ARGS = {
'task_emb_size': 5,
'code_dim': 4,
'n_embs': 3,
'decay': 0.99
}
ARGS = {
'x': torch.randn(2, 5)
}
if __name__ == '__main__':
model = VQTaskEncoder(**INIT_ARGS)
model.train()
output = model(**ARGS)
print("Output keys:", output.keys())
print("Encoded task emb shape:", output['encoded_task_emb'].shape)
print("Loss:", output['loss'].item())
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| add_full_stop |
Ran ran; our contract draft was wrong, not the code |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("e6ae5d7b6ce64ac3") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
def add_full_stop(s):
s = s.strip()
if s[-1].isalpha():
s += '.'
return s
ARGS = {
's': 'hello world'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| apply_rotary_pos_emb |
Ran ran; our contract draft was wrong, not the code |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("22f75dc2bea7b46e") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# Create small tensors for the function
batch_size = 2
hidden_dim = 4
# x: input tensor with shape (batch_size, hidden_dim)
x = torch.randn(batch_size, hidden_dim)
# cos: cosine positional embedding, same shape as x
cos = torch.randn(batch_size, hidden_dim)
# sin: sine positional embedding, same shape as x
sin = torch.randn(batch_size, hidden_dim)
ARGS = {
"x": x,
"cos": cos,
"sin": sin,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| cls_pool |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("ed175170e2628076") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# The function expects:
# 1. outputs: dict with 'last_hidden_state' key containing a tensor
# 2. attention_mask: tensor where we check if padding is right-aligned
# Create a small batch with batch_size=2, sequence_length=3, hidden_dim=4
batch_size = 2
seq_length = 3
hidden_dim = 4
# outputs dict with last_hidden_state tensor (batch_size, seq_length, hidden_dim)
outputs = {
'last_hidden_state': torch.randn(batch_size, seq_length, hidden_dim)
}
# attention_mask: (batch_size, seq_length)
# For right padding, all tokens should be attended to (1s followed by 0s if any)
# The check is: attention_mask[:, 0].sum() == attention_mask.shape[0]
# This means all first tokens should be 1 (padding_side='right')
attention_mask = torch.ones(batch_size, seq_length, dtype=torch.long)
ARGS = {
'outputs': outputs,
'attention_mask': attention_mask
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| embed_texts |
Ran ran, honoured the contract we drafted |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("895bcd8257169039") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from math import sqrt
torch.manual_seed(0)
# Small embedding model
class TinyEmbeddingModel(nn.Module):
def __init__(self, hidden_dim=4):
super().__init__()
self.hidden_dim = hidden_dim
self.embedding = nn.Embedding(100, hidden_dim)
self.linear = nn.Linear(hidden_dim, hidden_dim)
def forward(self, input_ids, attention_mask, output_hidden_states=True):
embedded = self.embedding(input_ids)
output = self.linear(embedded)
return type('Output', (), {
'last_hidden_state': output,
'hidden_states': [output]
})()
# Small tokenizer stand-in
class TinyTokenizer:
def __call__(self, texts, truncation=True, padding=True, max_length=8192, return_tensors='pt'):
batch_size = len(texts)
seq_length = 3
input_ids = torch.randint(1, 100, (batch_size, seq_length))
attention_mask = torch.ones(batch_size, seq_length)
return {
'input_ids': input_ids,
'attention_mask': attention_mask
}
# Simple pooling function
def pooling_fn(outputs, attention_mask):
hidden_states = outputs.last_hidden_state
mask_expanded = attention_mask.unsqueeze(-1).expand(hidden_states.size())
sum_hidden = (hidden_states * mask_expanded).sum(dim=1)
sum_mask = mask_expanded.sum(dim=1)
pooled = sum_hidden / sum_mask.clamp(min=1e-9)
return pooled
# Task description format function
def task_desc_format_fn(text):
return f"Task: {text}"
# Create instances
emb_model = TinyEmbeddingModel(hidden_dim=4)
emb_tokenizer = TinyTokenizer()
device = 'cpu'
# Sample texts
texts = ["text one", "text two"]
ARGS = {
'texts': texts,
'emb_model': emb_model,
'emb_tokenizer': emb_tokenizer,
'task_desc_format_fn': task_desc_format_fn,
'pooling_fn': pooling_fn,
'device': device,
'batch_size': None
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| embed_tokens |
Ran ran, honoured the contract we drafted |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("464adbe41baffcdb") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from math import sqrt
torch.manual_seed(0)
# Build a minimal embedding model that supports **kwargs and output_hidden_states
class MinimalEmbeddingModel(nn.Module):
def __init__(self, hidden_size=4):
super().__init__()
self.hidden_size = hidden_size
self.embedding = nn.Embedding(100, hidden_size)
self.linear = nn.Linear(hidden_size, hidden_size)
def forward(self, input_ids, attention_mask, output_hidden_states=False):
# input_ids: [batch_size, seq_len]
embedded = self.embedding(input_ids) # [batch_size, seq_len, hidden_size]
output = self.linear(embedded) # [batch_size, seq_len, hidden_size]
# Return an object with last_hidden_state attribute
class ModelOutput:
def __init__(self, last_hidden_state):
self.last_hidden_state = last_hidden_state
return ModelOutput(last_hidden_state=output)
# Build a minimal pooling function that takes outputs and attention_mask
def pooling_fn(outputs, attention_mask):
# outputs has last_hidden_state: [batch_size, seq_len, hidden_size]
# attention_mask: [batch_size, seq_len]
last_hidden = outputs.last_hidden_state # [batch_size, seq_len, hidden_size]
# Mean pooling: average over sequence dimension
# Expand mask to match hidden dimension
mask_expanded = attention_mask.unsqueeze(-1).expand(last_hidden.size()).float()
sum_hidden = (last_hidden * mask_expanded).sum(dim=1) # [batch_size, hidden_size]
sum_mask = mask_expanded.sum(dim=1) # [batch_size, hidden_size]
# Avoid division by zero
mean_pooled = sum_hidden / (sum_mask + 1e-9) # [batch_size, hidden_size]
return mean_pooled
# Create tokenized texts with batch size 2, sequence length 3
batch_size = 2
seq_len = 3
hidden_size = 4
tokenized_texts = {
'input_ids': torch.randint(0, 100, (batch_size, seq_len)),
'attention_mask': torch.ones((batch_size, seq_len), dtype=torch.long)
}
emb_model = MinimalEmbeddingModel(hidden_size=hidden_size)
device = 'cpu'
batch_size_param = 2
ARGS = {
'tokenized_texts': tokenized_texts,
'emb_model': emb_model,
'pooling_fn': pooling_fn,
'device': device,
'batch_size': batch_size_param
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_emb_model_and_fns |
Ran ran; our contract draft was wrong, not the code |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("a449079623eb8c90") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from unittest.mock import Mock, MagicMock
torch.manual_seed(0)
# Create a minimal AutoModel mock
class MockAutoModel(nn.Module):
def __init__(self):
super().__init__()
self.linear = nn.Linear(4, 4)
def forward(self, input_ids, attention_mask):
batch_size = input_ids.shape[0]
last_hidden_state = torch.randn(batch_size, 8, 4)
return {
'last_hidden_state': last_hidden_state,
'hidden_states': [torch.randn(batch_size, 8, 4) for _ in range(12)]
}
def eval(self):
return super().eval()
# Create a minimal tokenizer mock
class MockTokenizer:
def __init__(self):
self.pad_token_id = 0
self.eos_token_id = 1
self.padding_side = "right"
# Mock AutoModel.from_pretrained
def mock_from_pretrained(model_name, device_map=None, torch_dtype=None, trust_remote_code=False):
model = MockAutoModel()
return model
# Mock AutoTokenizer.from_pretrained
def mock_tokenizer_from_pretrained(model_name):
return MockTokenizer()
# Patch the imports at module level
import transformers
original_automodel = transformers.AutoModel
original_autotokenizer = transformers.AutoTokenizer
transformers.AutoModel.from_pretrained = staticmethod(mock_from_pretrained)
transformers.AutoTokenizer.from_pretrained = staticmethod(mock_tokenizer_from_pretrained)
ARGS = {
'emb_model_name': 'mock-gte-model',
'device': 'cpu'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_in_out_features |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("8cbf94593c76c7d4") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from peft import PeftConfig, PeftModel
from peft.tuners.tuners_utils import BaseTunerLayer
from unittest.mock import MagicMock, patch
torch.manual_seed(0)
# Create a mock PeftConfig
peft_config = MagicMock(spec=PeftConfig)
peft_config.target_modules = ['q_proj', 'v_proj']
# Create a mock BaseTunerLayer with base_layer as nn.Linear
class MockTunerLayer(nn.Module, BaseTunerLayer):
def __init__(self, name, in_feat, out_feat):
super().__init__()
self.name = name
self.in_features = in_feat
self.out_features = out_feat
self.base_layer = nn.Linear(in_feat, out_feat)
def forward(self, x):
return self.base_layer(x)
# Create a mock model with peft_config and named_modules
class MockPeftModel(nn.Module):
def __init__(self):
super().__init__()
self.peft_config = {'default': peft_config}
# Add tuner layers with distinct sizes
self.layer1_q_proj = MockTunerLayer('q_proj', 4, 5)
self.layer1_v_proj = MockTunerLayer('v_proj', 4, 6)
self.layer2_q_proj = MockTunerLayer('q_proj', 4, 5)
# Add a non-tuner layer that should be skipped
self.layer1_linear = nn.Linear(4, 5)
def named_modules(self):
yield 'layer1.q_proj', self.layer1_q_proj
yield 'layer1.v_proj', self.layer1_v_proj
yield 'layer2.q_proj', self.layer2_q_proj
yield 'layer1.linear', self.layer1_linear
model = MockPeftModel()
# Patch check_target_module_exists to return True for tuner layers
def mock_check_target_module_exists(config, module_name):
return 'q_proj' in module_name or 'v_proj' in module_name
with patch('peft.tuners.tuners_utils.check_target_module_exists', side_effect=mock_check_target_module_exists):
ARGS = {
'model': model,
'peft_config': None,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- peft, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_layers |
Ran ran; our contract draft was wrong, not the code |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("eb1fa1a3b994be5d") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned Sequential. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# The function checks if model has a 'model' attribute, and if so, recurses.
# Otherwise it returns model.layers
# So we need to create a model that has a .layers attribute
class SimpleModel(nn.Module):
def __init__(self):
super().__init__()
self.layers = nn.Sequential(
nn.Linear(4, 3),
nn.ReLU(),
nn.Linear(3, 2)
)
def forward(self, x):
return self.layers(x)
# Create a model instance
model = SimpleModel()
ARGS = {
'model': model
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Sequential — a Sequential output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a Sequential output the recorded ndim 1 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| get_pooling_fn |
Ran ran; our contract draft was wrong, not the code |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("f6bfe8f5c075f8d0") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned function. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from typing import Literal
torch.manual_seed(0)
# Test with 'last_token' pooling type
ARGS = {
'pooling_type': 'last_token'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type function — a function output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a function output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| last_token_pool |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("b6c4cb90c7bfeae1") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# Create a batch of 2 sequences with hidden dimension 4
batch_size = 2
seq_length = 3
hidden_dim = 4
# Create last_hidden_state tensor
last_hidden_state = torch.randn(batch_size, seq_length, hidden_dim)
# Create outputs dict with last_hidden_state
outputs = {
'last_hidden_state': last_hidden_state
}
# Create attention_mask - shape (batch_size, seq_length)
# Using right-padding scenario (not all 1s in last column)
attention_mask = torch.tensor([
[1, 1, 1],
[1, 1, 0]
], dtype=torch.long)
ARGS = {
'outputs': outputs,
'attention_mask': attention_mask
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| lora_state_dict_to_tensor_dict |
Ran ran; our contract draft was wrong, not the code |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("4eb352ce41bff2cc") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# Create a minimal LoRA state dict with the expected structure
# The function looks for keys containing 'layers.{layer_idx}' and 'lora_A' or 'lora_B'
lora_sd = {
'model.layers.0.self_attn.q_proj.lora_A.default': torch.randn(4, 3),
'model.layers.0.self_attn.q_proj.lora_B.default': torch.randn(3, 4),
'model.layers.1.self_attn.q_proj.lora_A.default': torch.randn(4, 3),
'model.layers.1.self_attn.q_proj.lora_B.default': torch.randn(3, 4),
'model.layers.0.mlp.up_proj.lora_A.default': torch.randn(4, 3),
'model.layers.0.mlp.up_proj.lora_B.default': torch.randn(3, 4),
'model.layers.1.mlp.up_proj.lora_A.default': torch.randn(4, 3),
'model.layers.1.mlp.up_proj.lora_B.default': torch.randn(3, 4),
}
# Target modules to look for in the state dict keys
target_modules = ['q_proj', 'up_proj']
# Layer indices to extract
layer_indices = [0, 1]
# Device (CPU only)
device = 'cpu'
ARGS = {
'lora_sd': lora_sd,
'target_modules': target_modules,
'layer_indices': layer_indices,
'device': device,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| lora_tensor_dict_to_param_dict |
Ran ran; our contract draft was wrong, not the code |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("1fc5218833c6c6ea") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned ParameterDict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Construct lora_tensor_dict with nested structure matching what the function expects
# The function expects a dict with keys 'A' and 'B', each containing dicts of tensors
lora_tensor_dict = {
'A': {
'layer1': torch.randn(4, 3),
'layer2': torch.randn(5, 2),
},
'B': {
'layer1': torch.randn(3, 4),
'layer2': torch.randn(2, 5),
}
}
requires_grad = True
ARGS = {
'lora_tensor_dict': lora_tensor_dict,
'requires_grad': requires_grad,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ParameterDict — a ParameterDict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a ParameterDict output the recorded ndim 1 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| zero_lora_param_dict |
Ran ran; our contract draft was wrong, not the code |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("4731a23d7470dc77") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ParameterDict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from torch import nn
torch.manual_seed(0)
# Define the arguments for zero_lora_param_dict
target_modules = ['q_proj', 'v_proj']
n_layers = 2
rank = 3
in_features = {'q_proj': 4, 'v_proj': 5}
out_features = {'q_proj': 6, 'v_proj': 7}
ARGS = {
'target_modules': target_modules,
'n_layers': n_layers,
'rank': rank,
'in_features': in_features,
'out_features': out_features
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ParameterDict — a ParameterDict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a ParameterDict output the recorded ndim 1 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| HyperModulator |
Not yet run unverified (harvested, no recorded run) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("c9a3604b537bfc0e") How it ran |
| TextToLoRA |
Not yet run unverified (harvested, no recorded run) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("81a8f2d7a29f3811") How it ran |
| _embed_tokens_single_batch |
Not yet run unverified (harvested, no recorded run) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("9a22c2784acf8fe5") How it ran |
| apply_lora_to_layers |
Not yet run unverified (harvested, no recorded run) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("937d38a0e976d855") How it ran |
| create_hypermod |
Not yet run unverified (harvested, no recorded run) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("6e23d59c5480daea") How it ran |
| get_init_peft_weights |
Not yet run unverified (harvested, no recorded run) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("b0fb60a036b8e6ee") How it ran |
| get_lora_module_names |
Not yet run unverified (harvested, no recorded run) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("7572df21036527e9") How it ran |
| get_model |
Not yet run unverified (harvested, no recorded run) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("122a565cbe1f2d83") How it ran |
| get_model_and_tokenizer |
Not yet run unverified (harvested, no recorded run) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("d38e50d952d93469") How it ran |
| get_peft_modules |
Not yet run unverified (harvested, no recorded run) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("4b6e8a0361945762") How it ran |
| get_tokenizer |
Not yet run unverified (harvested, no recorded run) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("f3fb84a5a6d3b065") How it ran |
| load_hypermod |
Not yet run unverified (harvested, no recorded run) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("177a916196a3d9b2") How it ran |
| load_hypermod_checkpoint |
Not yet run unverified (harvested, no recorded run) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("4dc184dbfb1660e4") How it ran |
| lora_forward |
Not yet run unverified (harvested, no recorded run) |
SakanaAI/doc-to-lora/src/ctx_to_lora/modeling/text_to_lora.py code served (permissive licence) · get_code("f8caf753f4a6cb6a") How it ran |