| ImageBOWEmbedding |
Ran ran on a synthesized input |
aalto-ai/sparse-compgen/grounded_compgen_research/models/imitation/conv_transformer.py code served (permissive licence) · get_code("ce9a198bf543d15d") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 4, dtype family float, shape (2, 32, 3, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
def initialize_parameters(m):
classname = m.__class__.__name__
if classname.find('Linear') != -1:
m.weight.data.normal_(0, 1)
m.weight.data *= 1 / torch.sqrt(m.weight.data.pow(2).sum(1, keepdim=True))
if m.bias is not None:
m.bias.data.fill_(0)
class ImageBOWEmbedding(nn.Module):
def __init__(self, max_value, n_channels, embedding_dim):
super().__init__()
self.max_value = max_value
self.embedding_dim = embedding_dim
self.embedding = nn.Embedding(n_channels * max_value, embedding_dim)
self.apply(initialize_parameters)
def forward(self, inputs):
offsets = torch.Tensor([0, self.max_value, 2 * self.max_value, 3 * self.max_value]).to(inputs.device)
offsetted = (inputs + offsets[None, None, None, :]).long()
each_embedding = self.embedding(offsetted)
each_embedding_flat = each_embedding.reshape(*each_embedding.shape[:-2], -1)
return each_embedding_flat.permute(0, 3, 1, 2)
INIT_ARGS = {
'max_value': 5,
'n_channels': 4,
'embedding_dim': 8,
}
ARGS = {
'inputs': torch.randint(0, 5, (2, 3, 4, 4), dtype=torch.float32),
}
if __name__ == '__main__':
model = ImageBOWEmbedding(**INIT_ARGS)
output = model(**ARGS)
print(f"Output shape: {output.shape}")
print(f"Output: {output}")
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 4, dtype family float, shape (2, 32, 3, 4) — shape (2, 32, 3, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| ImageBlock |
Ran ran on a synthesized input |
aalto-ai/sparse-compgen/grounded_compgen_research/models/imitation/conv_transformer.py code served (permissive licence) · get_code("f363048cbd07e14e") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 4, dtype family float, shape (2, 3, 5, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
def initialize_parameters(m):
classname = m.__class__.__name__
if classname.find('Linear') != -1:
m.weight.data.normal_(0, 1)
m.weight.data *= 1 / torch.sqrt(m.weight.data.pow(2).sum(1, keepdim=True))
if m.bias is not None:
m.bias.data.fill_(0)
class ImageBlock(nn.Module):
def __init__(self, in_channels, nheads):
super().__init__()
self.attn = nn.MultiheadAttention(in_channels, nheads, dropout=0.1)
self.bn1 = nn.LayerNorm(in_channels)
self.mlp = nn.Sequential(nn.Linear(in_channels, in_channels), nn.ReLU(), nn.Linear(in_channels, in_channels), nn.ReLU())
self.bn2 = nn.LayerNorm(in_channels)
self.apply(initialize_parameters)
def forward(self, x):
batch, height, width, features = x.shape
x = x.reshape(batch, height * width, features)
x_permuted = x.permute(1, 0, 2)
attended, _ = self.attn(x_permuted, x_permuted, x_permuted, need_weights=False)
attended = attended.permute(1, 0, 2)
attended_added = attended + x
attended_added_norm = self.bn1(attended_added)
attended_added_norm_mlp = self.mlp(attended_added_norm)
attended_added_norm_mlp_norm = self.bn2(attended_added_norm_mlp)
return attended_added_norm_mlp_norm.reshape(batch, height, width, features)
INIT_ARGS = {
'in_channels': 4,
'nheads': 2,
}
ARGS = {
'x': torch.randn(2, 3, 5, 4),
}
if __name__ == '__main__':
model = ImageBlock(**INIT_ARGS)
output = model(**ARGS)
print(f"Output shape: {output.shape}")
print(f"Expected shape: (2, 3, 5, 4)")
assert output.shape == (2, 3, 5, 4), f"Shape mismatch: {output.shape}"
print("Success!")
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 4, dtype family float, shape (2, 3, 5, 4) — shape (2, 3, 5, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| InstructionEncodingBlock |
Ran ran on a synthesized input |
aalto-ai/sparse-compgen/grounded_compgen_research/models/imitation/conv_transformer.py code served (permissive licence) · get_code("0f60d4b3cc7f7bdb") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
import torch
import torch.nn as nn
class PositionalEncoding(nn.Module):
"""This implementation is the same as in the Annotated transformer blog post
See https://nlp.seas.harvard.edu/2018/04/03/attention.html for more detail.
"""
def __init__(self, d_model, dropout=0.1, max_len=5000):
assert d_model % 2 == 0, 'd_model should be an even number.'
super(PositionalEncoding, self).__init__()
pe = torch.zeros(max_len, d_model)
position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)
div_term = torch.exp(torch.arange(0, d_model, 2).float() * (-np.log(10000.0) / d_model))
pe[:, 0::2] = torch.sin(position * div_term)
pe[:, 1::2] = torch.cos(position * div_term)
pe = pe.unsqueeze(0).transpose(0, 1)
self.register_buffer('pe', pe)
def forward(self, x):
return self.pe[:x.size(0), :]
class PositionalAddEncoding(nn.Module):
def __init__(self, d_model, dropout=0.1, max_len=5000):
super().__init__()
self.pos_encoding = PositionalEncoding(d_model, max_len=max_len)
self.dropout = nn.Dropout(p=dropout)
def forward(self, x):
pe = self.pos_encoding(x)
return self.dropout(x + pe)
class InstructionEncodingBlock(nn.Module):
def __init__(self, vocab_size, embedding_size, output_size):
super().__init__()
self.word_embedding = nn.Embedding(vocab_size, embedding_size)
self.gru = nn.GRU(embedding_size, embedding_size)
self.pos_encoding = PositionalAddEncoding(embedding_size)
def forward(self, sentence):
embedded = self.word_embedding(sentence.long())
init_hidden = torch.zeros_like(embedded[:, 0, :].unsqueeze(0))
embedded = embedded.permute(1, 0, 2)
gru_output, _ = self.gru(embedded, init_hidden)
pos_encoding = self.pos_encoding(gru_output)
pos_encoding = pos_encoding.permute(1, 0, 2)
gru_output = gru_output.permute(1, 0, 2)
return (gru_output, pos_encoding)
INIT_ARGS = {
'vocab_size': 100,
'embedding_size': 4,
'output_size': 8,
}
ARGS = {
'sentence': torch.randint(0, 100, (2, 3)),
}
if __name__ == '__main__':
model = InstructionEncodingBlock(**INIT_ARGS)
result = model(**ARGS)
print("Output shapes:")
print(f" gru_output: {result[0].shape}")
print(f" pos_encoding: {result[1].shape}")
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple — a tuple output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| ObservationBlock |
Ran ran on a synthesized input |
aalto-ai/sparse-compgen/grounded_compgen_research/models/imitation/conv_transformer.py code served (permissive licence) · get_code("6ff15de325f9d856") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 4, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
import numpy as np
def initialize_parameters(m):
classname = m.__class__.__name__
if classname.find('Linear') != -1:
m.weight.data.normal_(0, 1)
m.weight.data *= 1 / torch.sqrt(m.weight.data.pow(2).sum(1, keepdim=True))
if m.bias is not None:
m.bias.data.fill_(0)
class ImageBOWEmbedding(nn.Module):
def __init__(self, max_value, n_channels, embedding_dim):
super().__init__()
self.max_value = max_value
self.embedding_dim = embedding_dim
self.embedding = nn.Embedding(n_channels * max_value, embedding_dim)
self.apply(initialize_parameters)
def forward(self, inputs):
offsets = torch.Tensor([0, self.max_value, 2 * self.max_value, 3 * self.max_value]).to(inputs.device)
offsetted = (inputs + offsets[None, None, None, :]).long()
each_embedding = self.embedding(offsetted)
each_embedding_flat = each_embedding.reshape(*each_embedding.shape[:-2], -1)
return each_embedding_flat.permute(0, 3, 1, 2)
class ImageBlock(nn.Module):
def __init__(self, in_channels, nheads):
super().__init__()
self.attn = nn.MultiheadAttention(in_channels, nheads, dropout=0.1)
self.bn1 = nn.LayerNorm(in_channels)
self.mlp = nn.Sequential(nn.Linear(in_channels, in_channels), nn.ReLU(), nn.Linear(in_channels, in_channels), nn.ReLU())
self.bn2 = nn.LayerNorm(in_channels)
self.apply(initialize_parameters)
def forward(self, x):
batch, height, width, features = x.shape
x = x.reshape(batch, height * width, features)
x_permuted = x.permute(1, 0, 2)
attended, _ = self.attn(x_permuted, x_permuted, x_permuted, need_weights=False)
attended = attended.permute(1, 0, 2)
attended_added = attended + x
attended_added_norm = self.bn1(attended_added)
attended_added_norm_mlp = self.mlp(attended_added_norm)
attended_added_norm_mlp_norm = self.bn2(attended_added_norm_mlp)
return attended_added_norm_mlp_norm.reshape(batch, height, width, features)
class ObservationBlock(nn.Module):
def __init__(self, vocab_size, embedding_size, conv_dim, nheads):
super().__init__()
self.img_embeddings = ImageBOWEmbedding(16, 4, embedding_size)
self.conv2x2 = nn.Conv2d(in_channels=embedding_size * 4, out_channels=conv_dim, kernel_size=2, padding=1)
self.bn = nn.BatchNorm2d(conv_dim)
self.relu = nn.ReLU(inplace=True)
self.blocks = nn.Sequential(ImageBlock(conv_dim, nheads), ImageBlock(conv_dim, nheads), ImageBlock(conv_dim, nheads))
def forward(self, img):
embedded = self.img_embeddings(img)
conv_embedded = self.conv2x2(embedded)
conv_embedded_norm = self.bn(conv_embedded)
conv_embedded_norm_relu = self.relu(conv_embedded_norm)
conv_embedded_norm_relu = conv_embedded_norm_relu.permute(0, 2, 3, 1)
return self.blocks(conv_embedded_norm_relu)
INIT_ARGS = {
'vocab_size': 100,
'embedding_size': 3,
'conv_dim': 8,
'nheads': 2
}
ARGS = {
'img': torch.randint(0, 16, (2, 5, 7, 4))
}
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 4, dtype family float — no output shape is stated: observe_signature.py observed this class with its constructor sizes canonicalised where it could, which the node does not record, so its shape is not stated as this setup block's
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| PositionalAddEncoding |
Ran ran on a synthesized input |
aalto-ai/sparse-compgen/grounded_compgen_research/models/imitation/conv_transformer.py code served (permissive licence) · get_code("7a2b9fca729945a9") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float, shape (3, 2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
import torch
import torch.nn as nn
class PositionalEncoding(nn.Module):
"""This implementation is the same as in the Annotated transformer blog post
See https://nlp.seas.harvard.edu/2018/04/03/attention.html for more detail.
"""
def __init__(self, d_model, dropout=0.1, max_len=5000):
assert d_model % 2 == 0, 'd_model should be an even number.'
super(PositionalEncoding, self).__init__()
pe = torch.zeros(max_len, d_model)
position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)
div_term = torch.exp(torch.arange(0, d_model, 2).float() * (-np.log(10000.0) / d_model))
pe[:, 0::2] = torch.sin(position * div_term)
pe[:, 1::2] = torch.cos(position * div_term)
pe = pe.unsqueeze(0).transpose(0, 1)
self.register_buffer('pe', pe)
def forward(self, x):
return self.pe[:x.size(0), :]
class PositionalAddEncoding(nn.Module):
def __init__(self, d_model, dropout=0.1, max_len=5000):
super().__init__()
self.pos_encoding = PositionalEncoding(d_model, max_len=max_len)
self.dropout = nn.Dropout(p=dropout)
def forward(self, x):
pe = self.pos_encoding(x)
return self.dropout(x + pe)
INIT_ARGS = {
'd_model': 4,
'dropout': 0.1,
'max_len': 5000
}
ARGS = {
'x': torch.randn(3, 2, 4)
}
if __name__ == '__main__':
model = PositionalAddEncoding(**INIT_ARGS)
output = model(**ARGS)
print(f"Input shape: {ARGS['x'].shape}")
print(f"Output shape: {output.shape}")
print(f"Output:\n{output}")
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 3, dtype family float, shape (3, 2, 4) — shape (3, 2, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| PositionalEncoding |
Ran ran on a synthesized input |
aalto-ai/sparse-compgen/grounded_compgen_research/models/imitation/conv_transformer.py code served (permissive licence) · get_code("6d88134a57f82380") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float, shape (7, 1, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
import torch
import torch.nn as nn
class PositionalEncoding(nn.Module):
"""This implementation is the same as in the Annotated transformer blog post
See https://nlp.seas.harvard.edu/2018/04/03/attention.html for more detail.
"""
def __init__(self, d_model, dropout=0.1, max_len=5000):
assert d_model % 2 == 0, 'd_model should be an even number.'
super(PositionalEncoding, self).__init__()
pe = torch.zeros(max_len, d_model)
position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)
div_term = torch.exp(torch.arange(0, d_model, 2).float() * (-np.log(10000.0) / d_model))
pe[:, 0::2] = torch.sin(position * div_term)
pe[:, 1::2] = torch.cos(position * div_term)
pe = pe.unsqueeze(0).transpose(0, 1)
self.register_buffer('pe', pe)
def forward(self, x):
return self.pe[:x.size(0), :]
INIT_ARGS = {
'd_model': 4,
'dropout': 0.1,
'max_len': 100,
}
ARGS = {
'x': torch.randn(7, 2, 4),
}
if __name__ == '__main__':
model = PositionalEncoding(**INIT_ARGS)
output = model(**ARGS)
print(f"Output shape: {output.shape}")
print(f"Output:\n{output}")
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 3, dtype family float, shape (7, 1, 4) — shape (7, 1, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| RelationalBlock |
Ran ran on a synthesized input |
aalto-ai/sparse-compgen/grounded_compgen_research/models/imitation/conv_transformer.py code served (permissive licence) · get_code("11e6375c51d915d7") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
# INIT_ARGS for RelationalBlock.__init__(sentence_embedding_dim, pos_encoding_dim, conv_dim, nheads)
INIT_ARGS = {
'sentence_embedding_dim': 8,
'pos_encoding_dim': 6,
'conv_dim': 12,
'nheads': 3
}
# ARGS for RelationalBlock.forward(x)
# x is a tuple of (encoded_image, encoded_sentence, positional_encodings)
batch = 2
height = 4
width = 5
features = 12 # must match conv_dim
sentence_len = 7
encoded_image = torch.randn(batch, height, width, features)
encoded_sentence = torch.randn(batch, sentence_len, features)
positional_encodings = torch.randn(batch, sentence_len, features)
ARGS = {
'x': (encoded_image, encoded_sentence, positional_encodings)
}
- Setup block drafted by
- a language model, class construct-then-call prompt, model not recorded
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple — a tuple output has no array shape of its own; none is stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
|
| ConvTransformerModel |
Not yet run unverified (harvested, no recorded run) |
aalto-ai/sparse-compgen/grounded_compgen_research/models/imitation/conv_transformer.py code served (permissive licence) · get_code("86f07e914b4a1a28") How it ran |
| ProcessingTransformer |
Not yet run unverified (harvested, no recorded run) |
aalto-ai/sparse-compgen/grounded_compgen_research/models/imitation/conv_transformer.py code served (permissive licence) · get_code("e6c51e69ff1b0120") How it ran |
| initialize_parameters |
Not yet run unverified (harvested, no recorded run) |
aalto-ai/sparse-compgen/grounded_compgen_research/models/imitation/conv_transformer.py code served (permissive licence) · get_code("6125d291452ba72a") How it ran |