| Trainer_attention_grad_compute_loss |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
thu-nics/moa/MoA/attention/sparse_mask.py code served (permissive licence) · get_code("b8258f95889e4952") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from torch.nn import Module
from collections import namedtuple
torch.manual_seed(0)
# Mock PeftModel for type checking
class PeftModel:
pass
# Create a minimal model that outputs attentions
class MockModel(nn.Module):
def __init__(self):
super().__init__()
self.linear = nn.Linear(4, 4)
def forward(self, input_ids=None, attention_mask=None, labels=None, output_attentions=False):
batch_size = 2
seq_len = 3
num_heads = 2
# Create mock attention tensors with 4D shape (batch, num_heads, seq_len, seq_len)
attention_1 = torch.randn(batch_size, num_heads, seq_len, seq_len)
attention_2 = torch.randn(batch_size, num_heads, seq_len, seq_len)
# Create loss
loss = torch.tensor(1.5, requires_grad=True)
# Create output object
output = namedtuple('Output', ['loss', 'attentions'])
return output(loss=loss, attentions=(attention_1, attention_2))
# Create a minimal label smoother
class MockLabelSmoother:
def __call__(self, outputs, labels, shift_labels=False):
return torch.tensor(1.5, requires_grad=True)
# Create a minimal args object
class MockArgs:
def __init__(self):
self.past_index = -1
# Create a minimal trainer object
class MockTrainer:
def __init__(self):
self.label_smoother = MockLabelSmoother()
self.args = MockArgs()
self.atten_map = None
self.attention_grads = None
self._past = None
# Create input tensors
inputs = {
'input_ids': torch.randint(0, 100, (2, 3)),
'attention_mask': torch.ones((2, 3)),
}
model = MockModel()
trainer = MockTrainer()
ARGS = {
'self': trainer,
'model': model,
'inputs': inputs,
'return_outputs': False,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| count_distinct_elements_per_row |
Ran ran; our contract draft was wrong, not the code |
thu-nics/moa/MoA/attention/density_calculation.py code served (permissive licence) · get_code("29414013ce12967c") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 1, dtype family int, shape (2,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# Create a small 2D integer tensor with some repeated values
# Shape: (2, 4) - 2 rows, 4 columns
lut = torch.tensor([
[1, 2, 1, 3],
[5, 5, 5, 7]
], dtype=torch.int32)
ARGS = {
"lut": lut
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 1, dtype family int, shape (2,) — shape (2,) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| find_subtensor_position |
Ran ran; our contract draft was wrong, not the code |
thu-nics/moa/MoA/dataset/utils.py code served (permissive licence) · get_code("061f1b20e4c107be") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# Create a small sub_tensor to search for
# Shape: (len_sub,) - let's use length 2
sub_tensor = torch.tensor([1, 2])
# Create a main_tensor with batch dimension
# Shape: (batch, len_main) - let's use batch=2, len_main=5
main_tensor = torch.tensor([
[0, 1, 2, 3, 1],
[1, 2, 4, 1, 2]
])
ARGS = {
"sub_tensor": sub_tensor,
"main_tensor": main_tensor
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list — a list output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| gen_band_pattern |
Ran ran, violated the contract we drafted |
thu-nics/moa/MoA/attention/pattern.py code served (permissive licence) · get_code("149e4b82b7620c4b") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 4, dtype family float, shape (2, 3, 4, 5) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from torch import Tensor
torch.manual_seed(0)
# Small test values
num_query = 4
num_key = 5
band_width_int = 3
num_layer = 2
num_head = 3
ARGS = {
"num_query": num_query,
"num_key": num_key,
"band_width": band_width_int,
"dtype": torch.bool,
"num_layer": num_layer,
"num_head": num_head,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 4, dtype family float, shape (2, 3, 4, 5) — shape (2, 3, 4, 5) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| gen_block_sparse_layout_mask |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
thu-nics/moa/MoA/attention/sparse_mask.py code served (permissive licence) · get_code("6d1389e6b2d85469") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from typing import Optional, Tuple
torch.manual_seed(0)
# Create a small matrix tensor
# Shape: [keep_dim=2, matrix_size_x=8, matrix_size_y=12]
# This allows for num_block_x=2, num_block_y=3 with block sizes of 4x4
matrix = torch.randn(2, 8, 12)
# Number of blocks in x and y directions
num_block_x = 2
num_block_y = 3
# Number of blocks to keep per keep dimension
# Shape: [2] to match keep_dim
block_keep_num = torch.tensor([2, 2])
# Row balance flag
row_balance = False
# Custom aggregation function (optional)
def custom_agg_func(block_matrix: torch.Tensor) -> torch.Tensor:
"""Aggregate block matrix by taking mean over last two dimensions."""
return torch.mean(block_matrix, dim=[-1, -2])
block_aggregation_func = custom_agg_func
# Causal mask flag
is_causal = False
ARGS = {
"matrix": matrix,
"num_block_x": num_block_x,
"num_block_y": num_block_y,
"block_keep_num": block_keep_num,
"row_balance": row_balance,
"block_aggregation_func": block_aggregation_func,
"is_causal": is_causal,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple — a tuple output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| gen_global_pattern |
Ran ran; our contract draft was wrong, not the code |
thu-nics/moa/MoA/attention/pattern.py code served (permissive licence) · get_code("7e1d14c04aa0dc2d") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 4, dtype family float, shape (1, 1, 8, 8) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
ARGS = {
"num_query": 8,
"num_key": 8,
"key_pos": [0, 2, 5],
"dtype": torch.bool,
"num_layer": 1,
"num_head": 1,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 4, dtype family float, shape (1, 1, 8, 8) — shape (1, 1, 8, 8) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| head_index_to_split_index |
Ran ran; our contract draft was wrong, not the code |
thu-nics/moa/MoA/kernels/flash_decoding_moa.py code served (permissive licence) · get_code("2d84082c203b2b52") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 1, dtype family int, shape (5,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# head_index should be a tensor of shape (num_heads + 1)
# representing the start index of each head
# Let's create a simple example with 3 heads
head_index = torch.tensor([0, 10, 20, 30], dtype=torch.long)
# split_size is an integer
split_size = 7
ARGS = {
"head_index": head_index,
"split_size": split_size,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 1, dtype family int, shape (5,) — shape (5,) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| hotpotQA_to_multi_round_qa |
Ran ran; our contract draft was wrong, not the code |
thu-nics/moa/MoA/dataset/convert.py code served (permissive licence) · get_code("21ce5f5f68d41dad") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
# Construct the entry dictionary based on the hotpotQA data structure
entry = {
"answer": "This is the answer",
"context": {
"sentences": [["Sent 1"], ["Sent 21", "Sent 22"]],
"title": ["Title1", "Title 2"]
},
"id": "000001",
"level": "medium",
"question": "What is the answer?",
"supporting_facts": {
"sent_id": [0, 1],
"title": ["Title1", "Title 2"]
},
"type": "comparison"
}
ARGS = {
"entry": entry
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| layout_to_lut |
Ran ran; our contract draft was wrong, not the code |
thu-nics/moa/MoA/attention/convert.py code served (permissive licence) · get_code("2866d7c2b62c260d") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# Create a small layout tensor with shape (layer, num_heads, num_block, num_block)
# Using distinct sizes: 2 layers, 3 heads, 4 blocks
layout = torch.zeros((2, 3, 4, 4), dtype=torch.bool)
# Fill with some sparse patterns so we have non-zero values
# For layer 0, head 0
layout[0, 0, 0, [0, 2]] = True
layout[0, 0, 1, [1, 3]] = True
layout[0, 0, 2, [0, 1, 2]] = True
layout[0, 0, 3, [1, 2]] = True
# For layer 0, head 1
layout[0, 1, 0, [1, 3]] = True
layout[0, 1, 1, [0, 2]] = True
layout[0, 1, 2, [1, 3]] = True
layout[0, 1, 3, [0, 2]] = True
# For layer 0, head 2
layout[0, 2, 0, [0, 1]] = True
layout[0, 2, 1, [2, 3]] = True
layout[0, 2, 2, [0, 3]] = True
layout[0, 2, 3, [1, 2]] = True
# For layer 1, head 0
layout[1, 0, 0, [0, 1, 2]] = True
layout[1, 0, 1, [1, 3]] = True
layout[1, 0, 2, [0, 2]] = True
layout[1, 0, 3, [0, 1]] = True
# For layer 1, head 1
layout[1, 1, 0, [1, 2]] = True
layout[1, 1, 1, [0, 1, 3]] = True
layout[1, 1, 2, [2, 3]] = True
layout[1, 1, 3, [0, 2]] = True
# For layer 1, head 2
layout[1, 2, 0, [0, 3]] = True
layout[1, 2, 1, [1, 2]] = True
layout[1, 2, 2, [0, 1, 2]] = True
layout[1, 2, 3, [1, 3]] = True
ARGS = {
'layout': layout
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple — a tuple output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| lut_to_permutation |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
thu-nics/moa/MoA/attention/permutation_utils.py code served (permissive licence) · get_code("e6a764e7c48e4500") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# The function expects a list of lookup tables (LUTs)
# Each LUT is a 3D tensor of shape (num_heads, height, width)
# The function serializes each head, sorts them, and clusters identical heads
num_heads = 32
num_key_value_groups = 1
# Create sample LUT tensors
# Shape: (num_heads, height, width) where height and width are arbitrary dimensions
lut1 = torch.randn(num_heads, 4, 5)
lut2 = torch.randn(num_heads, 4, 5)
# Make some heads identical to ensure clustering works
# Heads 0-15 get one pattern, heads 16-31 get another pattern
lut1[0:16] = torch.ones(16, 4, 5) * 0.5
lut1[16:32] = torch.ones(16, 4, 5) * 1.5
lut2[0:16] = torch.ones(16, 4, 5) * 0.3
lut2[16:32] = torch.ones(16, 4, 5) * 1.3
lut_list = [lut1, lut2]
ARGS = {
'lut_list': lut_list,
'num_heads': num_heads,
'num_key_value_groups': num_key_value_groups,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple — a tuple output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| mask_to_layout |
Ran ran; our contract draft was wrong, not the code |
thu-nics/moa/MoA/attention/convert.py code served (permissive licence) · get_code("8633f899e4d7407c") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 3, dtype family float, shape (2, 4, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn.functional as F
torch.manual_seed(0)
# Create a small mask tensor
# Shape: (batch=2, token_length=8, token_length=8)
mask = torch.randint(0, 2, (2, 8, 8)).float()
# block_size parameter
block_size = 2
ARGS = {
"mask": mask,
"block_size": block_size,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 3, dtype family float, shape (2, 4, 4) — shape (2, 4, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| matrix_reserve_score |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
thu-nics/moa/MoA/attention/sparse_mask.py code served (permissive licence) · get_code("0bfc9650b5be48e8") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 1, dtype family float, shape (2,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from torch import Tensor
torch.manual_seed(0)
# Create small tensors with distinct dimensions
# matrix shape: [keep_dim, matrix_size_x, matrix_size_y]
# Using keep_dim=2, matrix_size_x=4, matrix_size_y=3
keep_dim = 2
matrix_size_x = 4
matrix_size_y = 3
# Create matrix tensor with random values
matrix = torch.randn(keep_dim, matrix_size_x, matrix_size_y)
# Create mask tensor with binary values (0 and 1)
mask = torch.randint(0, 2, (keep_dim, matrix_size_x, matrix_size_y)).float()
# is_causal: boolean flag
is_causal = False
# norm_score: boolean flag
norm_score = False
ARGS = {
"matrix": matrix,
"mask": mask,
"is_causal": is_causal,
"norm_score": norm_score,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 1, dtype family float, shape (2,) — shape (2,) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| moa_config_to_cache_config |
Ran ran; our contract draft was wrong, not the code |
thu-nics/moa/MoA/attention/cache_utils.py code served (permissive licence) · get_code("1ada47e15f33f461") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import pandas as pd
import math
# Create a minimal moa_config dictionary
# The function expects 'alphas' and 'betas' keys
# alphas and betas are nested lists: [layer][head]
# We'll create a small 2-layer, 3-head config
moa_config = {
'alphas': [
[10.5, 20.3, 15.7], # layer 0, 3 heads
[12.1, 18.9, 22.4], # layer 1, 3 heads
],
'betas': [
[0.1, 0.15, 0.12], # layer 0, 3 heads
[0.11, 0.14, 0.13], # layer 1, 3 heads
]
}
ARGS = {
'moa_config': moa_config,
'seq_len': 512,
'max_new_token': 1024,
'sink_size': 64,
'minimum_cache_size': 128,
'split_size': 64,
'verbose': True,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- pandas — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| moa_config_to_permutation |
Ran ran; our contract draft was wrong, not the code |
thu-nics/moa/MoA/attention/permutation_utils.py code served (permissive licence) · get_code("2cd7b8febe087a7d") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# Create a minimal moa_config dictionary with alphas and betas
# alphas and betas are lists of lists (one per layer), where each inner list contains values for each head
moa_config = {
'alphas': [
[1, 2, 1, 3], # Layer 0: 4 heads with alpha values
[2, 1, 2], # Layer 1: 3 heads with alpha values
],
'betas': [
[5, 4, 5, 6], # Layer 0: 4 heads with beta values
[3, 2, 3], # Layer 1: 3 heads with beta values
]
}
ARGS = {
'moa_config': moa_config
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple — a tuple output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| multiNews_to_multi_round_qa |
Ran ran; our contract draft was wrong, not the code |
thu-nics/moa/MoA/dataset/convert.py code served (permissive licence) · get_code("3dca86fbcf33d8a4") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
# Construct a valid entry for the multiNews_to_multi_round_qa function
entry = {
'document': 'some line val\nanother line\nthird line of context',
'summary': '– target val line'
}
ARGS = {
'entry': entry
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| permute_lut |
Ran ran; our contract draft was wrong, not the code |
thu-nics/moa/MoA/attention/permutation_utils.py code served (permissive licence) · get_code("77491a1736272578") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (32, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# Construct arguments for permute_lut function
lut = torch.randn(32, 4) # Shape: (num_heads, feature_dim)
permutation = torch.tensor([1, 0, 3, 2, 5, 4, 7, 6, 9, 8, 11, 10, 13, 12, 15, 14, 17, 16, 19, 18, 21, 20, 23, 22, 25, 24, 27, 26, 29, 28, 31, 30]) # Permutation for 32 heads
num_heads = 32
ARGS = {
"lut": lut,
"permutation": permutation,
"num_heads": num_heads
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 2, dtype family float, shape (32, 4) — shape (32, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| qasper_to_multi_round_qa |
Ran ran; our contract draft was wrong, not the code |
thu-nics/moa/MoA/dataset/convert.py code served (permissive licence) · get_code("89ec645a857ef9af") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
ARGS = {
'entry': {
'id': 'paper_001',
'title': 'Understanding Neural Networks',
'abstract': 'This paper explores the fundamental principles of neural network architectures and their applications in modern machine learning.',
'full_text': {
'paragraphs': [
[
'Neural networks are computational models inspired by biological neural networks.',
'They consist of interconnected nodes organized in layers.'
],
[
'Deep learning has revolutionized computer vision and natural language processing.',
'Convolutional networks are particularly effective for image recognition tasks.'
]
],
'section_name': ['Introduction', 'Background']
},
'qas': {
'question': [
'What are neural networks?',
'How have neural networks impacted machine learning?'
],
'question_id': ['q_001', 'q_002'],
'question_writer': ['writer_001', 'writer_002'],
'nlp_background': [True, False],
'topic_background': [True, True],
'paper_read': [True, False],
'search_query': ['neural networks basics', 'deep learning applications'],
'answers': [
{
'annotation_id': ['ann_001', 'ann_002'],
'answer': [
{
'unanswerable': False,
'extractive_spans': ['computational models inspired by biological neural networks'],
'yes_no': False,
'free_form_answer': 'Neural networks are computational models consisting of interconnected nodes organized in layers.',
'evidence': ['Neural networks are computational models inspired by biological neural networks.'],
'highlighted_evidence': ['Neural networks are computational models']
},
{
'unanswerable': False,
'extractive_spans': [],
'yes_no': False,
'free_form_answer': 'They are inspired by biological systems and organized in layers.',
'evidence': ['They consist of interconnected nodes organized in layers.'],
'highlighted_evidence': ['interconnected nodes organized in layers']
}
],
'worker_id': ['worker_001', 'worker_002']
},
{
'annotation_id': ['ann_003'],
'answer': [
{
'unanswerable': False,
'extractive_spans': ['revolutionized computer vision and natural language processing'],
'yes_no': False,
'free_form_answer': 'Deep learning has revolutionized computer vision and natural language processing.',
'evidence': ['Deep learning has revolutionized computer vision and natural language processing.'],
'highlighted_evidence': ['revolutionized computer vision']
}
],
'worker_id': ['worker_003']
}
]
}
}
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| streamingllm_attention_density |
Ran ran, honoured the contract we drafted |
thu-nics/moa/MoA/attention/density_calculation.py code served (permissive licence) · get_code("cd7b0c5e2a16d0cd") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
ARGS = {
"global_size": 4,
"band_size": 1024,
"kv_seq_len": 8192,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| streamingllm_kv_cache_density |
Ran ran, honoured the contract we drafted |
thu-nics/moa/MoA/attention/density_calculation.py code served (permissive licence) · get_code("758a42acae59b91f") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
ARGS = {
"global_size": 4,
"band_size": 1024,
"kv_seq_len": 8192,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| block_sparse_to_dense |
Not yet run unverified (harvested, no recorded run) |
thu-nics/moa/MoA/attention/convert.py code served (permissive licence) · get_code("98558d05575a5d9f") How it ran |
| gen_causal_pattern |
Not yet run unverified (harvested, no recorded run) |
thu-nics/moa/MoA/attention/pattern.py code served (permissive licence) · get_code("ef170fe22d7a924e") How it ran |
| update_model_function |
Not yet run unverified (harvested, no recorded run) |
thu-nics/moa/MoA/models/interface.py code served (permissive licence) · get_code("e55fe24f97290938") How it ran |