| Attention |
Ran ran on a synthesized input · constructed an object; no result was computed |
lucidrains/g-mlp-pytorch/g_mlp_pytorch/g_mlp_pytorch.py code served (permissive licence) · get_code("444b2d5ae8422156") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Attention: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from torch import nn
torch.manual_seed(0)
# Define ARGS with all parameters for Attention.__init__
ARGS = {
'dim_in': 8,
'dim_out': 6,
'dim_inner': 4,
'causal': False,
}
# Create an instance to verify it works
attention = nn.Attention(**ARGS) if hasattr(nn, 'Attention') else None
# Create a small input tensor for testing
# Shape: (batch=2, sequence_length=3, dim_in=8)
x = torch.randn(2, 3, ARGS['dim_in'])
# If you want to actually run forward pass:
# attention_module = Attention(**ARGS)
# output = attention_module(x)
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Attention — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Attention output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| Attention |
Ran ran on a synthesized input |
lucidrains/g-mlp-gpt/g_mlp_gpt/g_mlp_gpt.py code served (permissive licence) · get_code("400cf93bc9a76062") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float, shape (2, 3, 5) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from torch import nn, einsum
class Attention(nn.Module):
def __init__(self, dim_in, dim_out, dim_inner):
super().__init__()
self.scale = dim_inner ** (-0.5)
self.to_qkv = nn.Linear(dim_in, dim_inner * 3, bias=False)
self.to_out = nn.Linear(dim_inner, dim_out)
def forward(self, x):
device = x.device
q, k, v = self.to_qkv(x).chunk(3, dim=-1)
sim = einsum('b i d, b j d -> b i j', q, k) * self.scale
mask = torch.ones(sim.shape[-2:], device=device).triu(1).bool()
sim.masked_fill_(mask[None, ...], -torch.finfo(q.dtype).max)
attn = sim.softmax(dim=-1)
out = einsum('b i j, b j d -> b i d', attn, v)
return self.to_out(out)
INIT_ARGS = {
'dim_in': 4,
'dim_out': 5,
'dim_inner': 6,
}
ARGS = {
'x': torch.randn(2, 3, 4),
}
if __name__ == '__main__':
attention = Attention(**INIT_ARGS)
output = attention(**ARGS)
print(f"Output shape: {output.shape}")
print(f"Expected shape: (2, 3, 5)")
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 3, dtype family float, shape (2, 3, 5) — shape (2, 3, 5) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| CausalLocalSGU |
Ran ran on a synthesized input |
lucidrains/g-mlp-gpt/g_mlp_gpt/g_mlp_gpt.py code served (permissive licence) · get_code("3bb856991018170a") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float, shape (2, 64, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from math import ceil
# Import the class
from math import ceil
import torch
import torch.nn.functional as F
from torch import nn, einsum
from einops import rearrange, repeat
def exists(val):
return val is not None
def pad_to_multiple(tensor, multiple, dim=-1, value=0):
seqlen = tensor.shape[dim]
m = seqlen / multiple
if m.is_integer():
return tensor
remainder = ceil(m) * multiple - seqlen
pad_offset = (0,) * (-1 - dim) * 2
return F.pad(tensor, (*pad_offset, 0, remainder), value=value)
class CausalLocalSGU(nn.Module):
def __init__(self, dim, dim_seq, init_eps=0.001, heads=4, window=128, act=nn.Identity()):
super().__init__()
dim_out = dim // 2
self.norm = nn.LayerNorm(dim_out)
self.heads = heads
self.window = window
self.weight = nn.Parameter(torch.zeros(heads, window, window * 2))
self.bias = nn.Parameter(torch.zeros(heads, window))
init_eps /= window
nn.init.uniform_(self.weight, -init_eps, init_eps)
nn.init.constant_(self.bias, 1.0)
self.act = act
self.register_buffer('mask', ~torch.ones(window, window * 2).triu_(window + 1).bool())
def forward(self, x, gate_res=None):
device, n, h, w = (x.device, x.shape[1], self.heads, self.window)
res, gate = x.chunk(2, dim=-1)
gate = pad_to_multiple(gate, w, dim=-2)
gate = rearrange(gate, 'b (w n) d -> b w n d', n=w)
gate = self.norm(gate)
gate = F.pad(gate, (0, 0, 0, 0, 1, 0), value=0.0)
gate = torch.cat((gate[:, :-1], gate[:, 1:]), dim=2)
weight, bias = (self.weight, self.bias)
weight = weight * self.mask[None, ...].int().float()
gate = rearrange(gate, 'b w n (h d) -> b w h n d', h=h)
gate = einsum('b w h n d, h m n -> b w h m d', gate, weight)
gate = gate + rearrange(bias, 'h n -> () () h n ()')
gate = rearrange(gate, 'b w h n d -> b w n (h d)')
gate = rearrange(gate, 'b w n d -> b (w n) d')
gate = gate[:, :n]
if exists(gate_res):
gate = gate + gate_res
return self.act(gate) * res
INIT_ARGS = {
'dim': 8,
'dim_seq': 256,
'init_eps': 0.001,
'heads': 2,
'window': 32,
'act': nn.Identity()
}
ARGS = {
'x': torch.randn(2, 64, 8),
'gate_res': None
}
if __name__ == '__main__':
model = CausalLocalSGU(**INIT_ARGS)
output = model(**ARGS)
print(f"Output shape: {output.shape}")
print(f"Expected shape: {ARGS['x'].shape}")
print("Success!")
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- einops, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 3, dtype family float, shape (2, 64, 4) — shape (2, 64, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| CausalSGU |
Ran ran on a synthesized input |
lucidrains/g-mlp-gpt/g_mlp_gpt/g_mlp_gpt.py code served (permissive licence) · get_code("7a6075afb6e7a2c2") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float, shape (2, 3, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from torch import nn, einsum
from einops import rearrange, repeat
def exists(val):
return val is not None
class CausalSGU(nn.Module):
def __init__(self, dim, dim_seq, init_eps=0.001, heads=4, act=nn.Identity()):
super().__init__()
dim_out = dim // 2
self.norm = nn.LayerNorm(dim_out)
self.heads = heads
self.weight = nn.Parameter(torch.zeros(heads, dim_seq, dim_seq))
self.bias = nn.Parameter(torch.zeros(heads, dim_seq))
init_eps /= dim_seq
nn.init.uniform_(self.weight, -init_eps, init_eps)
nn.init.constant_(self.bias, 1.0)
self.act = act
self.register_buffer('mask', ~torch.ones(dim_seq, dim_seq).triu_(1).bool())
def forward(self, x, gate_res=None):
device, n, h = (x.device, x.shape[1], self.heads)
res, gate = x.chunk(2, dim=-1)
gate = self.norm(gate)
weight, bias = (self.weight, self.bias)
weight, bias = (weight[:, :n, :n], bias[:, :n])
weight = weight * self.mask[None, :n, :n].int().float()
gate = rearrange(gate, 'b n (h d) -> b h n d', h=h)
gate = einsum('b h n d, h m n -> b h m d', gate, weight)
gate = gate + rearrange(bias, 'h n -> () h n ()')
gate = rearrange(gate, 'b h n d -> b n (h d)')
if exists(gate_res):
gate = gate + gate_res
return self.act(gate) * res
INIT_ARGS = {
'dim': 8,
'dim_seq': 3,
'init_eps': 0.001,
'heads': 2,
'act': nn.Identity()
}
ARGS = {
'x': torch.randn(2, 3, 8),
'gate_res': None
}
if __name__ == '__main__':
model = CausalSGU(**INIT_ARGS)
output = model(**ARGS)
print(f"Output shape: {output.shape}")
print(f"Output: {output}")
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- einops, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 3, dtype family float, shape (2, 3, 4) — shape (2, 3, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| GELU |
Ran ran on a synthesized input |
NydiaAI/g-mlp-tensorflow/gmlp/gmlp.py pointer only (licence: NONE) · get_code("36c054162a22371c") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, NydiaAI/g-mlp-tensorflow when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| GatingMlpBlock |
Ran ran on a synthesized input · constructed an object; no result was computed |
antonyvigouret/Pay-Attention-to-MLPs/models.py code served (permissive licence) · get_code("cb304156fe457ec9") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, GatingMlpBlock: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Parameters for GatingMlpBlock
d_model = 8
d_ffn = 16
seq_len = 4
survival_prob = 0.8
ARGS = {
"d_model": d_model,
"d_ffn": d_ffn,
"seq_len": seq_len,
"survival_prob": survival_prob
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type GatingMlpBlock — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a GatingMlpBlock output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| LocalAttention |
Ran ran on a synthesized input |
lucidrains/g-mlp-gpt/g_mlp_gpt/g_mlp_gpt.py code served (permissive licence) · get_code("94e83172b0355df5") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float, shape (2, 12, 6) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from math import ceil
INIT_ARGS = {
'dim_in': 4,
'dim_inner': 8,
'dim_out': 6,
'window': 4,
}
ARGS = {
'x': torch.randn(2, 12, 4),
}
if __name__ == '__main__':
from math import ceil
import torch
import torch.nn.functional as F
from torch import nn, einsum
from einops import rearrange, repeat
def pad_to_multiple(tensor, multiple, dim=-1, value=0):
seqlen = tensor.shape[dim]
m = seqlen / multiple
if m.is_integer():
return tensor
remainder = ceil(m) * multiple - seqlen
pad_offset = (0,) * (-1 - dim) * 2
return F.pad(tensor, (*pad_offset, 0, remainder), value=value)
class LocalAttention(nn.Module):
def __init__(self, dim_in, dim_inner, dim_out, window=128):
super().__init__()
self.scale = dim_inner ** (-0.5)
self.window = window
self.to_qkv = nn.Linear(dim_in, dim_inner * 3, bias=False)
self.to_out = nn.Linear(dim_inner, dim_out)
def forward(self, x):
b, n, *_, device, w = (*x.shape, x.device, self.window)
x = pad_to_multiple(x, w, dim=-2, value=0.0)
q, k, v = self.to_qkv(x).chunk(3, dim=-1)
window_fn = lambda t: rearrange(t, 'b (w n) d -> b w n d', n=w)
q, k, v = map(window_fn, (q, k, v))
k, v = map(lambda t: F.pad(t, (0, 0, 0, 0, 1, 0)), (k, v))
k, v = map(lambda t: torch.cat((k[:, :-1], k[:, 1:]), dim=2), (k, v))
sim = einsum('b w i d, b w j d -> b w i j', q, k) * self.scale
buckets, i, j = sim.shape[-3:]
mask_value = -torch.finfo(sim.dtype).max
mask = torch.ones(i, j, device=device).triu_(j - i + 1).bool()
mask = repeat(mask, 'i j -> () u i j', u=buckets)
sim.masked_fill_(mask, mask_value)
attn = sim.softmax(dim=-1)
out = einsum('b w i j, b w j d -> b w i d', attn, v)
out = rearrange(out, 'b w n d -> b (w n) d')
out = self.to_out(out[:, :n])
return out
model = LocalAttention(**INIT_ARGS)
output = model(**ARGS)
print(f"Output shape: {output.shape}")
print(f"Output: {output}")
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- einops, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 3, dtype family float, shape (2, 12, 6) — shape (2, 12, 6) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| PreShiftTokens |
Ran ran on a synthesized input · constructed an object; no result was computed |
lucidrains/g-mlp-pytorch/g_mlp_pytorch/g_mlp_pytorch.py code served (permissive licence) · get_code("c78c09b8919ac748") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, PreShiftTokens: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Create a minimal fn that accepts x and **kwargs
# The forward method just needs to accept x and return something
class DummyModule(nn.Module):
def forward(self, x, **kwargs):
return x
# Create the dummy function module
fn = DummyModule()
# shifts: tuple of shift amounts for each segment
shifts = (1, -1, 0)
# ARGS dict with all parameters
ARGS = {
"shifts": shifts,
"fn": fn
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type PreShiftTokens — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a PreShiftTokens output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| Residual |
Ran ran on a synthesized input · constructed an object; no result was computed |
xmu-xiaoma666/External-Attention-pytorch/model/mlp/g_mlp.py code served (permissive licence) · get_code("9cceceb3671335d7") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Residual: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# The Residual module wraps a function fn and adds a residual connection.
# It calls self.fn(x) and adds x to it, so fn must be a module that:
# - takes input x (a tensor)
# - returns output of same shape as x
# The forward pass does: return self.fn(x) + x
# Create a simple function module that preserves shape
class IdentityModule(nn.Module):
def forward(self, x):
return x
# Create the fn argument - a simple module
fn = IdentityModule()
ARGS = {
"fn": fn
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Residual — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Residual output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| Residual |
Ran ran on a synthesized input · constructed an object; no result was computed |
lucidrains/g-mlp-pytorch/g_mlp_pytorch/g_mlp_pytorch.py code served (permissive licence) · get_code("4b858823b3eccdd7") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Residual: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from torch import nn
torch.manual_seed(0)
# The Residual class wraps a function/module and adds a residual connection
# It calls self.fn(x) and adds x to the result
# So fn must be a callable (nn.Module) that takes x and returns something
# with the same shape as x
class SimpleLinear(nn.Module):
"""A simple linear module that preserves input shape"""
def __init__(self, dim):
super().__init__()
self.linear = nn.Linear(dim, dim)
def forward(self, x):
return self.linear(x)
# Create a small function/module to wrap
fn = SimpleLinear(dim=4)
ARGS = {
'fn': fn
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Residual — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Residual output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| Residual |
Ran ran on a synthesized input |
NydiaAI/g-mlp-tensorflow/gmlp/gmlp.py pointer only (licence: NONE) · get_code("f9f191324f0a26fb") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, NydiaAI/g-mlp-tensorflow when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| SpatialGatingUnit |
Ran ran on a synthesized input · constructed an object; no result was computed |
xmu-xiaoma666/External-Attention-pytorch/model/mlp/g_mlp.py code served (permissive licence) · get_code("9c81afbca1a2169c") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, SpatialGatingUnit: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from torch import nn
torch.manual_seed(0)
ARGS = {
"dim": 8,
"len_sen": 5
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type SpatialGatingUnit — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a SpatialGatingUnit output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| SpatialGatingUnit |
Ran ran on a synthesized input |
antonyvigouret/Pay-Attention-to-MLPs/models.py code served (permissive licence) · get_code("1356a744f3730d9c") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
INIT_ARGS = {
'd_ffn': 8,
'seq_len': 4,
}
ARGS = {
'x': torch.randn(2, 4, 8),
}
if __name__ == '__main__':
model = nn.Module()
model.__class__ = type('SpatialGatingUnit', (nn.Module,), {
'__init__': lambda self, d_ffn, seq_len: (
super(type(self), self).__init__(),
setattr(self, 'norm', nn.LayerNorm(d_ffn // 2)),
setattr(self, 'proj', nn.Linear(seq_len, seq_len))
),
'forward': lambda self, x: (
lambda u, v: u * (
self.proj(
self.norm(v).permute(0, 2, 1)
).permute(0, 2, 1)
)
)(*x.chunk(2, dim=-1))
})
model = type('SpatialGatingUnit', (nn.Module,), {
'__init__': lambda self, d_ffn, seq_len: (
super(type(self), self).__init__(),
setattr(self, 'norm', nn.LayerNorm(d_ffn // 2)),
setattr(self, 'proj', nn.Linear(seq_len, seq_len))
),
'forward': lambda self, x: (
lambda u, v: u * (
self.proj(
self.norm(v).permute(0, 2, 1)
).permute(0, 2, 1)
)
)(*x.chunk(2, dim=-1))
})(**INIT_ARGS)
output = model(**ARGS)
print(f"Output shape: {output.shape}")
print(f"Output: {output}")
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 3, dtype family float — no output shape is stated: observe_signature.py observed this class with its constructor sizes canonicalised where it could, which the node does not record, so its shape is not stated as this setup block's
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| SpatialGatingUnit |
Ran ran on a synthesized input · constructed an object; no result was computed |
jaketae/g-mlp/g_mlp/core.py code served (permissive licence) · get_code("d300d8fa28344b6c") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, SpatialGatingUnit: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# The function creates a SpatialGatingUnit which expects:
# - d_ffn: the feature dimension (used in LayerNorm and Conv1d)
# - seq_len: the sequence length (used in Conv1d)
#
# In forward(), it expects input x with shape (batch, seq_len, d_ffn)
# which gets chunked into u and v along dim=-1
d_ffn = 8 # feature dimension
seq_len = 5 # sequence length
ARGS = {
"d_ffn": d_ffn,
"seq_len": seq_len,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type SpatialGatingUnit — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a SpatialGatingUnit output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| SpatialGatingUnit |
Ran ran on a synthesized input · constructed an object; no result was computed |
cvolcy/gmlp-tf/gmlp.py pointer only (licence: NONE) · get_code("2098e31f03598b8e") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, cvolcy/gmlp-tf when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| SpatialGatingUnit |
Ran ran on a synthesized input · constructed an object; no result was computed |
lucidrains/g-mlp-pytorch/g_mlp_pytorch/g_mlp_pytorch.py code served (permissive licence) · get_code("6d43444c56cad017") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, SpatialGatingUnit: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Define ARGS dict with all parameters for SpatialGatingUnit
ARGS = {
'dim': 8,
'dim_seq': 4,
'causal': False,
'act': nn.GELU(),
'heads': 2,
'init_eps': 0.001,
'circulant_matrix': False,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- einops, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type SpatialGatingUnit — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a SpatialGatingUnit output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| SpatialGatingUnit |
Ran ran on a synthesized input |
leaderj1001/Bag-of-MLP/model.py code served (permissive licence) · get_code("f84eabb818f0b01a") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float, shape (2, 3, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
class TinyAttention(nn.Module):
def __init__(self, d_ffn, d_attn=64):
super(TinyAttention, self).__init__()
self.qkv = nn.Linear(d_ffn, 3 * d_attn)
self.d_attn = d_attn
self.softmax = nn.Softmax(dim=-1)
self.out = nn.Linear(d_attn, d_ffn // 2)
def forward(self, x):
qkv = self.qkv(x)
q, k, v = torch.chunk(qkv, 3, dim=-1)
energy = torch.bmm(q, k.transpose(1, 2)) / self.d_attn ** 0.5
attn = self.softmax(energy)
out = torch.bmm(attn, v)
out = self.out(out)
return out
class SpatialGatingUnit(nn.Module):
def __init__(self, seq_len, d_model, attn=True):
super(SpatialGatingUnit, self).__init__()
self.layer_norm = nn.LayerNorm(d_model // 2)
self.spatial = nn.Conv1d(seq_len, seq_len, kernel_size=1)
self.is_attn = attn
if self.is_attn:
self.attn = TinyAttention(d_model)
def forward(self, x):
if self.is_attn:
shortcut = x
shortcut = self.attn(shortcut)
u, v = torch.chunk(x, 2, dim=-1)
v = self.layer_norm(v)
v = self.spatial(v)
if self.is_attn:
v += shortcut
return u * v
INIT_ARGS = {
'seq_len': 3,
'd_model': 8,
'attn': True
}
ARGS = {
'x': torch.randn(2, 3, 8)
}
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 3, dtype family float, shape (2, 3, 4) — shape (2, 3, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| TinyAttention |
Ran ran on a synthesized input |
leaderj1001/Bag-of-MLP/model.py code served (permissive licence) · get_code("ccc79f7e78680ff2") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float, shape (2, 3, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
class TinyAttention(nn.Module):
def __init__(self, d_ffn, d_attn=64):
super(TinyAttention, self).__init__()
self.qkv = nn.Linear(d_ffn, 3 * d_attn)
self.d_attn = d_attn
self.softmax = nn.Softmax(dim=-1)
self.out = nn.Linear(d_attn, d_ffn // 2)
def forward(self, x):
qkv = self.qkv(x)
q, k, v = torch.chunk(qkv, 3, dim=-1)
energy = torch.bmm(q, k.transpose(1, 2)) / self.d_attn ** 0.5
attn = self.softmax(energy)
out = torch.bmm(attn, v)
out = self.out(out)
return out
INIT_ARGS = {
'd_ffn': 8,
'd_attn': 64,
}
ARGS = {
'x': torch.randn(2, 3, 8),
}
if __name__ == '__main__':
model = TinyAttention(**INIT_ARGS)
output = model(**ARGS)
print(f"Output shape: {output.shape}")
print(f"Output: {output}")
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 3, dtype family float, shape (2, 3, 4) — shape (2, 3, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| cast_tuple |
Ran ran, violated the contract we drafted |
lucidrains/g-mlp-gpt/g_mlp_gpt/g_mlp_gpt.py code served (permissive licence) · get_code("092c152ffc42d497") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned tuple, ndim 1, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# Analyzing the function:
# def cast_tuple(val, num):
# return (val,) * num if not isinstance(val, tuple) else val
#
# Parameters:
# - val: can be any value or a tuple
# - num: an integer used for tuple repetition
ARGS = {
"val": 42,
"num": 3
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family int — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| check_sizes |
Ran ran, honoured the contract we drafted |
this paper's copy was not recorded; identical code first harvested from jaketae/res-mlp pointer only · get_code("27cbea18b00696e9") How it ranThe body of this sample is withheld: this paper's copy of the code was not recorded (its link names no repository or source path), so no licence was read for it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| dropout_layers |
Ran ran; our contract draft was wrong, not the code |
lucidrains/g-mlp-pytorch/g_mlp_pytorch/g_mlp_pytorch.py code served (permissive licence) · get_code("1d39b4f9d1d36f92") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from random import randrange
torch.manual_seed(0)
# Create a list of simple layers (nn.Module instances)
layers = [
nn.Linear(4, 3),
nn.ReLU(),
nn.Linear(3, 2),
nn.Dropout(0.5)
]
# Define probability of survival
prob_survival = 0.7
ARGS = {
"layers": layers,
"prob_survival": prob_survival
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| exist |
Ran ran, violated the contract we drafted |
xmu-xiaoma666/External-Attention-pytorch/model/mlp/g_mlp.py code served (permissive licence) · get_code("62c69ec0b60162c6") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
ARGS = {
"x": None
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| exists |
Ran ran, violated the contract we drafted |
this paper's copy was not recorded; identical code first harvested from ThomasMrY/VCT pointer only; the stored copy get_code serves, from ThomasMrY/VCT, is pointer only (licence: NONE) · get_code("aa5486a3650902d8") How it ranThe body of this sample is withheld: this paper's copy of the code was not recorded (its link names no repository or source path), so no licence was read for it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| gMLP |
Ran ran on a synthesized input · constructed an object; no result was computed |
xmu-xiaoma666/External-Attention-pytorch/model/mlp/g_mlp.py code served (permissive licence) · get_code("b9fa9cafe23e12fb") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, gMLP: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from torch import nn
from collections import OrderedDict
torch.manual_seed(0)
# Define ARGS dict with all parameters for gMLP
ARGS = {
'num_tokens': 100,
'len_sen': 49,
'dim': 8,
'd_ff': 16,
'num_layers': 2
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type gMLP — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a gMLP output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| gMLP |
Ran ran on a synthesized input · constructed an object; no result was computed |
antonyvigouret/Pay-Attention-to-MLPs/models.py code served (permissive licence) · get_code("5a39fbe0ee0a5529") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, gMLP: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Parameters for gMLP
d_model = 4
d_ffn = 8
seq_len = 3
n_blocks = 2
prob_0_L = [1.0, 0.5]
ARGS = {
"d_model": d_model,
"d_ffn": d_ffn,
"seq_len": seq_len,
"n_blocks": n_blocks,
"prob_0_L": prob_0_L,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type gMLP — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a gMLP output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| gMLP |
Ran ran on a synthesized input · constructed an object; no result was computed |
jaketae/g-mlp/g_mlp/core.py code served (permissive licence) · get_code("2949429d580ede4b") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, gMLP: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
ARGS = {
"d_model": 4,
"d_ffn": 8,
"seq_len": 3,
"num_layers": 2,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type gMLP — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a gMLP output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| gMLP |
Ran ran on a synthesized input |
leaderj1001/Bag-of-MLP/model.py code served (permissive licence) · get_code("bfbcb5bb0523c7c8") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 2, dtype family float, shape (2, 5) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
import torch.nn.functional as F
class TinyAttention(nn.Module):
def __init__(self, d_ffn, d_attn=64):
super(TinyAttention, self).__init__()
self.qkv = nn.Linear(d_ffn, 3 * d_attn)
self.d_attn = d_attn
self.softmax = nn.Softmax(dim=-1)
self.out = nn.Linear(d_attn, d_ffn // 2)
def forward(self, x):
qkv = self.qkv(x)
q, k, v = torch.chunk(qkv, 3, dim=-1)
energy = torch.bmm(q, k.transpose(1, 2)) / self.d_attn ** 0.5
attn = self.softmax(energy)
out = torch.bmm(attn, v)
out = self.out(out)
return out
class SpatialGatingUnit(nn.Module):
def __init__(self, seq_len, d_model, attn=True):
super(SpatialGatingUnit, self).__init__()
self.layer_norm = nn.LayerNorm(d_model // 2)
self.spatial = nn.Conv1d(seq_len, seq_len, kernel_size=1)
self.is_attn = attn
if self.is_attn:
self.attn = TinyAttention(d_model)
def forward(self, x):
if self.is_attn:
shortcut = x
shortcut = self.attn(shortcut)
u, v = torch.chunk(x, 2, dim=-1)
v = self.layer_norm(v)
v = self.spatial(v)
if self.is_attn:
v += shortcut
return u * v
class gMLPBlock(nn.Module):
def __init__(self, seq_len, d_model, d_ffn):
super(gMLPBlock, self).__init__()
self.layer_norm = nn.LayerNorm(d_model)
self.channel1 = nn.Linear(d_model, d_ffn)
self.sgu = SpatialGatingUnit(seq_len, d_ffn)
self.channel2 = nn.Linear(d_ffn // 2, d_model)
def forward(self, x):
shortcut = x
out = self.layer_norm(x)
out = self.channel1(out)
out = F.gelu(out)
out = self.sgu(out)
out = self.channel2(out)
return out + shortcut
class gMLP(nn.Module):
def __init__(self, seq_len, d_model, d_ffn, patch=32, N=12, n_classes=10):
super(gMLP, self).__init__()
self.d_model = d_model
self.embedding = nn.Conv2d(3, d_model, kernel_size=patch, stride=patch)
self.layers = nn.ModuleList([gMLPBlock(seq_len, d_model, d_ffn) for _ in range(N)])
self.out = nn.Linear(d_model, n_classes)
def forward(self, x):
out = self.embedding(x).permute(0, 2, 3, 1).view(x.size(0), -1, self.d_model)
for layer in self.layers:
out = layer(out)
out = out.mean(1)
out = self.out(out)
return out
INIT_ARGS = {
'seq_len': 4,
'd_model': 8,
'd_ffn': 16,
'patch': 32,
'N': 2,
'n_classes': 5,
}
ARGS = {
'x': torch.randn(2, 3, 64, 64),
}
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 2, dtype family float, shape (2, 5) — shape (2, 5) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| gMLPBlock |
Ran ran on a synthesized input · constructed an object; no result was computed |
jaketae/g-mlp/g_mlp/core.py code served (permissive licence) · get_code("3f7743323edaa181") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, gMLPBlock: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Parameters for gMLPBlock
d_model = 4 # Model dimension
d_ffn = 8 # Feed-forward network dimension (must be even for chunk(2))
seq_len = 3 # Sequence length
ARGS = {
'd_model': d_model,
'd_ffn': d_ffn,
'seq_len': seq_len
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type gMLPBlock — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a gMLPBlock output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| gMLPBlock |
Ran ran on a synthesized input · constructed an object; no result was computed |
lucidrains/g-mlp-pytorch/g_mlp_pytorch/g_mlp_pytorch.py code served (permissive licence) · get_code("63d8448fca0c6967") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, gMLPBlock: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
ARGS = {
'dim': 4,
'dim_ff': 8,
'seq_len': 3,
'heads': 1,
'attn_dim': None,
'causal': False,
'act': nn.Identity(),
'circulant_matrix': False,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- einops, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type gMLPBlock — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a gMLPBlock output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| gMLPBlock |
Ran ran on a synthesized input |
leaderj1001/Bag-of-MLP/model.py code served (permissive licence) · get_code("af39a3c7ed4a3a0c") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float, shape (2, 5, 8) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
import torch.nn.functional as F
class TinyAttention(nn.Module):
def __init__(self, d_ffn, d_attn=64):
super(TinyAttention, self).__init__()
self.qkv = nn.Linear(d_ffn, 3 * d_attn)
self.d_attn = d_attn
self.softmax = nn.Softmax(dim=-1)
self.out = nn.Linear(d_attn, d_ffn // 2)
def forward(self, x):
qkv = self.qkv(x)
q, k, v = torch.chunk(qkv, 3, dim=-1)
energy = torch.bmm(q, k.transpose(1, 2)) / self.d_attn ** 0.5
attn = self.softmax(energy)
out = torch.bmm(attn, v)
out = self.out(out)
return out
class SpatialGatingUnit(nn.Module):
def __init__(self, seq_len, d_model, attn=True):
super(SpatialGatingUnit, self).__init__()
self.layer_norm = nn.LayerNorm(d_model // 2)
self.spatial = nn.Conv1d(seq_len, seq_len, kernel_size=1)
self.is_attn = attn
if self.is_attn:
self.attn = TinyAttention(d_model)
def forward(self, x):
if self.is_attn:
shortcut = x
shortcut = self.attn(shortcut)
u, v = torch.chunk(x, 2, dim=-1)
v = self.layer_norm(v)
v = self.spatial(v)
if self.is_attn:
v += shortcut
return u * v
class gMLPBlock(nn.Module):
def __init__(self, seq_len, d_model, d_ffn):
super(gMLPBlock, self).__init__()
self.layer_norm = nn.LayerNorm(d_model)
self.channel1 = nn.Linear(d_model, d_ffn)
self.sgu = SpatialGatingUnit(seq_len, d_ffn)
self.channel2 = nn.Linear(d_ffn // 2, d_model)
def forward(self, x):
shortcut = x
out = self.layer_norm(x)
out = self.channel1(out)
out = F.gelu(out)
out = self.sgu(out)
out = self.channel2(out)
return out + shortcut
INIT_ARGS = {
'seq_len': 5,
'd_model': 8,
'd_ffn': 16
}
ARGS = {
'x': torch.randn(2, 5, 8)
}
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 3, dtype family float, shape (2, 5, 8) — shape (2, 5, 8) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| gMLPLayer |
Ran ran on a synthesized input · constructed an object; no result was computed |
cvolcy/gmlp-tf/gmlp.py pointer only (licence: NONE) · get_code("683c46ded1adfd74") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, cvolcy/gmlp-tf when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| pad_to_multiple |
Ran ran; our contract draft was wrong, not the code |
lucidrains/g-mlp-gpt/g_mlp_gpt/g_mlp_gpt.py code served (permissive licence) · get_code("df3be65159852cd8") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned Tensor, ndim 2, dtype family float, shape (2, 6) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn.functional as F
from math import ceil
torch.manual_seed(0)
# Create a small tensor with shape (2, 4) - batch size 2, sequence length 4
tensor = torch.randn(2, 4)
# multiple: the target multiple to pad to
multiple = 3
# dim: dimension to pad along (default -1 means last dimension)
dim = -1
# value: padding value (default 0)
value = 0
ARGS = {
"tensor": tensor,
"multiple": multiple,
"dim": dim,
"value": value,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 2, dtype family float, shape (2, 6) — shape (2, 6) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| pair |
Ran ran, violated the contract we drafted |
this paper's copy was not recorded; identical code first harvested from lucidrains/res-mlp-pytorch pointer only · get_code("e6f9d3b5ad66ee20") How it ranThe body of this sample is withheld: this paper's copy of the code was not recorded (its link names no repository or source path), so no licence was read for it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded. |
| shift |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
lucidrains/g-mlp-pytorch/g_mlp_pytorch/g_mlp_pytorch.py code served (permissive licence) · get_code("2fdf59e29c6d95cc") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn.functional as F
torch.manual_seed(0)
# Create a small tensor for the shift function
# The function uses F.pad which expects a tensor
# Shape: (batch=2, features=4)
t = torch.randn(2, 4)
# amount parameter: number of positions to shift
amount = 1
# mask parameter: optional, can be None or a tensor
mask = None
ARGS = {
"t": t,
"amount": amount,
"mask": mask,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| Attention |
Not yet run unverified (harvested, no recorded run) |
lucidrains/mlp-gpt-jax/mlp_gpt_jax/mlp_gpt_jax.py code served (permissive licence) · get_code("01675e7e7c713147") How it ran |
| PreNorm |
Not yet run unverified (harvested, no recorded run) |
NydiaAI/g-mlp-tensorflow/gmlp/gmlp.py pointer only (licence: NONE) · get_code("2c295e59b874c818") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, NydiaAI/g-mlp-tensorflow when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded. |
| SGU |
Not yet run unverified (harvested, no recorded run) |
lucidrains/mlp-gpt-jax/mlp_gpt_jax/mlp_gpt_jax.py code served (permissive licence) · get_code("1b9ff0230e5d08d3") How it ran |
| SpatialGatingUnit |
Not yet run unverified (harvested, no recorded run) |
NydiaAI/g-mlp-tensorflow/gmlp/gmlp.py pointer only (licence: NONE) · get_code("51216a9ccabc7c3f") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, NydiaAI/g-mlp-tensorflow when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded. |
| gMLP |
Not yet run unverified (harvested, no recorded run) |
lucidrains/mlp-gpt-jax/mlp_gpt_jax/mlp_gpt_jax.py code served (permissive licence) · get_code("c1658eda58b9fa37") How it ran |
| gMLP |
Not yet run unverified (harvested, no recorded run) |
lucidrains/g-mlp-pytorch/g_mlp_pytorch/g_mlp_pytorch.py code served (permissive licence) · get_code("0330d8c0ae7e5def") How it ran |
| gMLP |
Not yet run unverified (harvested, no recorded run) |
radi-cho/gatedtabtransformer/gated_tab_transformer/g_mlp.py code served (permissive licence) · get_code("2e5cc43cba18a48d") How it ran |
| gMLP |
Not yet run unverified (harvested, no recorded run) |
NydiaAI/g-mlp-tensorflow/gmlp/gmlp.py pointer only (licence: NONE) · get_code("211dccfc0c5af2fa") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, NydiaAI/g-mlp-tensorflow when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded. |
| gMLPBlock |
Not yet run unverified (harvested, no recorded run) |
lucidrains/g-mlp-gpt/g_mlp_gpt/g_mlp_gpt.py code served (permissive licence) · get_code("d16a844fcf910249") How it ran |
| gMLPBlock |
Not yet run unverified (harvested, no recorded run) |
NydiaAI/g-mlp-tensorflow/gmlp/gmlp.py pointer only (licence: NONE) · get_code("a5c1690311b9aad2") How it ranThe body of this sample is withheld: GitHub detected no licence in this paper's repository, NydiaAI/g-mlp-tensorflow when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded. |