| BaseTextTaskHelper |
Ran ran on a synthesized input · constructed an object; no result was computed |
g8a9/ferret/ferret/explainers/gradient.py code served (permissive licence) · get_code("f815bed681b134dc") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, BaseTextTaskHelper: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from transformers.tokenization_utils_base import BatchEncoding
torch.manual_seed(0)
# Create a minimal tokenizer stand-in
class MinimalTokenizer:
def __init__(self):
self.vocab_size = 100
self.model_max_length = 512
self.pad_token_id = 0
def __call__(self, text, return_tensors=None, truncation=False, padding=None, **kwargs):
if isinstance(text, str):
text = [text]
elif isinstance(text, list) and isinstance(text[0], str):
pass
else:
text = list(text)
batch_size = len(text)
seq_len = 5
input_ids = torch.randint(1, self.vocab_size, (batch_size, seq_len))
attention_mask = torch.ones(batch_size, seq_len)
if return_tensors == 'pt':
return BatchEncoding({
'input_ids': input_ids,
'attention_mask': attention_mask
})
return {
'input_ids': input_ids,
'attention_mask': attention_mask
}
def convert_ids_to_tokens(self, ids, **kwargs):
return [f"token_{int(i)}" for i in ids]
# Create a minimal model config
class MinimalConfig:
def __init__(self):
self.hidden_size = 4
self.vocab_size = 100
self.id2label = {0: "label_0", 1: "label_1"}
self.device = 'cpu'
# Create a minimal model
class MinimalModel(nn.Module):
def __init__(self):
super().__init__()
self.config = MinimalConfig()
self.device = 'cpu'
self.embedding = nn.Embedding(100, 4)
self.linear = nn.Linear(4, 2)
def get_input_embeddings(self):
return self.embedding
def forward(self, input_ids=None, inputs_embeds=None, attention_mask=None, output_hidden_states=False):
if inputs_embeds is not None:
hidden = inputs_embeds
else:
hidden = self.embedding(input_ids)
logits = self.linear(hidden)
class Output:
def __init__(self, logits):
self.logits = logits
return Output(logits)
model = MinimalModel()
tokenizer = MinimalTokenizer()
ARGS = {
'model': model,
'tokenizer': tokenizer
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch, tqdm, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type BaseTextTaskHelper — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a BaseTextTaskHelper output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| Explanation |
Ran ran on a synthesized input · constructed an object; no result was computed |
g8a9/ferret/ferret/explainers/gradient.py code served (permissive licence) · get_code("7be99d7ee78f65a2") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Explanation: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
from dataclasses import dataclass
from typing import Optional
@dataclass
class Explanation:
"""Generic class to represent an Explanation"""
text: str
tokens: str
scores: np.array
explainer: str
target_pos_idx: int
helper_type: str
target_token_pos_idx: Optional[int] = None
target: Optional[str] = None
target_token: Optional[str] = None
ARGS = {
"text": "The quick brown fox jumps over the lazy dog",
"tokens": "The quick brown fox jumps over the lazy dog",
"scores": np.array([0.1, 0.2, 0.15, 0.3, 0.25, 0.05, 0.12, 0.18, 0.08]),
"explainer": "attention",
"target_pos_idx": 3,
"helper_type": "gradient",
"target_token_pos_idx": 2,
"target": "fox",
"target_token": "brown"
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Explanation — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Explanation output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| SequenceClassificationHelper |
Ran ran on a synthesized input · constructed an object; no result was computed |
g8a9/ferret/ferret/explainers/gradient.py code served (permissive licence) · get_code("6d7956d91e40f00a") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, SequenceClassificationHelper: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from transformers.tokenization_utils_base import BatchEncoding
torch.manual_seed(0)
# Create a minimal tokenizer stand-in
class MinimalTokenizer:
def __init__(self):
self.vocab_size = 100
self.model_max_length = 128
self.pad_token_id = 0
self.eos_token_id = 2
def __call__(self, text, return_tensors=None, truncation=False, padding=None, **kwargs):
if isinstance(text, str):
text = [text]
# Create simple token ids (batch_size, seq_len)
batch_size = len(text)
seq_len = 8
input_ids = torch.randint(3, 100, (batch_size, seq_len))
attention_mask = torch.ones(batch_size, seq_len)
if return_tensors == 'pt':
return BatchEncoding({
'input_ids': input_ids,
'attention_mask': attention_mask
})
return BatchEncoding({
'input_ids': input_ids,
'attention_mask': attention_mask
})
def convert_ids_to_tokens(self, ids, **kwargs):
return [f"token_{i}" for i in ids]
# Create a minimal model config stand-in
class MinimalConfig:
def __init__(self):
self.id2label = {0: "label_0", 1: "label_1"}
self.num_labels = 2
self.hidden_size = 4
self.vocab_size = 100
# Create a minimal model
class MinimalModel(nn.Module):
def __init__(self):
super().__init__()
self.config = MinimalConfig()
self.device = torch.device('cpu')
self.embedding = nn.Embedding(100, 4)
self.classifier = nn.Linear(4, 2)
def get_input_embeddings(self):
return self.embedding
def forward(self, input_ids=None, inputs_embeds=None, attention_mask=None, output_hidden_states=False):
if input_ids is not None:
hidden_states = self.embedding(input_ids)
else:
hidden_states = inputs_embeds
# Simple pooling: take mean over sequence dimension
pooled = hidden_states.mean(dim=1)
logits = self.classifier(pooled)
class Output:
def __init__(self, logits, hidden_states_list=None):
self.logits = logits
self.hidden_states = hidden_states_list
return Output(logits, [hidden_states] if output_hidden_states else None)
# Create model and tokenizer
model = MinimalModel()
tokenizer = MinimalTokenizer()
ARGS = {
'model': model,
'tokenizer': tokenizer
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch, tqdm, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type SequenceClassificationHelper — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a SequenceClassificationHelper output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| TokenClassificationHelper |
Ran ran on a synthesized input · constructed an object; no result was computed |
g8a9/ferret/ferret/explainers/gradient.py code served (permissive licence) · get_code("51a529a9451f6518") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, TokenClassificationHelper: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from transformers.tokenization_utils_base import BatchEncoding
torch.manual_seed(0)
# Create a minimal model that satisfies the usage in the code
class MinimalModel(nn.Module):
def __init__(self):
super().__init__()
self.config = type('Config', (), {
'id2label': {0: 'LABEL_0', 1: 'LABEL_1'},
'hidden_size': 4,
'vocab_size': 100,
})()
self.device = 'cpu'
self.embeddings = nn.Embedding(100, 4)
def get_input_embeddings(self):
return self.embeddings
def forward(self, input_ids=None, inputs_embeds=None, attention_mask=None, output_hidden_states=False):
if inputs_embeds is not None:
logits = torch.randn(inputs_embeds.shape[0], 2)
else:
logits = torch.randn(input_ids.shape[0], input_ids.shape[1], 2)
output = type('Output', (), {
'logits': logits,
'hidden_states': None,
})()
return output
# Create a minimal tokenizer
class MinimalTokenizer:
def __init__(self):
self.pad_token_id = 0
self.eos_token_id = 1
def __call__(self, text, return_tensors=None, truncation=False, padding=None, **kwargs):
if isinstance(text, str):
text = [text]
# Create dummy token ids
max_len = 5
input_ids = [[2, 3, 4, 0, 0] for _ in text]
attention_mask = [[1, 1, 1, 0, 0] for _ in text]
if return_tensors == 'pt':
input_ids = torch.tensor(input_ids)
attention_mask = torch.tensor(attention_mask)
return BatchEncoding({
'input_ids': input_ids,
'attention_mask': attention_mask,
})
def convert_ids_to_tokens(self, ids, **kwargs):
token_map = {0: '[PAD]', 1: '[EOS]', 2: '[CLS]', 3: 'token1', 4: 'token2'}
if isinstance(ids, torch.Tensor):
ids = ids.tolist()
return [token_map.get(id, f'token_{id}') for id in ids]
model = MinimalModel()
tokenizer = MinimalTokenizer()
ARGS = {
'model': model,
'tokenizer': tokenizer,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch, tqdm, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type TokenClassificationHelper — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a TokenClassificationHelper output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| ZeroShotTextClassificationHelper |
Ran ran on a synthesized input · constructed an object; no result was computed |
g8a9/ferret/ferret/explainers/gradient.py code served (permissive licence) · get_code("6ee6b2a8a337c1f7") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ZeroShotTextClassificationHelper: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
from transformers.tokenization_utils_base import BatchEncoding
from abc import ABC, abstractmethod
torch.manual_seed(0)
# Create a minimal tokenizer stand-in
class MinimalTokenizer:
def __init__(self):
self.vocab_size = 100
self.model_max_length = 512
def __call__(self, text, return_tensors=None, truncation=False, padding=None, **kwargs):
if isinstance(text, str):
text = [text]
elif isinstance(text, list) and len(text) > 0 and isinstance(text[0], str):
pass
else:
text = [str(t) for t in text]
batch_size = len(text)
seq_len = 8
input_ids = torch.randint(0, self.vocab_size, (batch_size, seq_len))
attention_mask = torch.ones(batch_size, seq_len)
return BatchEncoding({
'input_ids': input_ids,
'attention_mask': attention_mask
})
def convert_ids_to_tokens(self, ids, **kwargs):
return [f"token_{int(id)}" for id in ids]
# Create a minimal model
class MinimalModel(nn.Module):
def __init__(self):
super().__init__()
self.embedding = nn.Embedding(100, 4)
self.classifier = nn.Linear(4, 3)
self.device_attr = torch.device('cpu')
# Config with required attributes
class Config:
id2label = {0: "negative", 1: "neutral", 2: "positive"}
self.config = Config()
@property
def device(self):
return self.device_attr
def get_input_embeddings(self):
return self.embedding
def forward(self, input_ids=None, attention_mask=None, inputs_embeds=None, output_hidden_states=False):
if inputs_embeds is not None:
hidden_states = inputs_embeds
else:
hidden_states = self.embedding(input_ids)
logits = self.classifier(hidden_states.mean(dim=1))
class Output:
pass
out = Output()
out.logits = logits
if output_hidden_states:
out.hidden_states = (hidden_states,)
return out
# Create instances
model = MinimalModel()
tokenizer = MinimalTokenizer()
# Build ARGS dictionary for BaseTextTaskHelper.__init__
ARGS = {
'model': model,
'tokenizer': tokenizer
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch, tqdm, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ZeroShotTextClassificationHelper — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ZeroShotTextClassificationHelper output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| distance |
Ran ran on a synthesized input |
ink-usc/dig/monotonic_paths.py code served (permissive licence) · get_code("83a6815c4f83e018") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float64, ndim 0, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
import torch
# The function distance() takes two array-like arguments A and B
# and computes the Euclidean distance between them.
# Both need to be numpy arrays or compatible with numpy operations.
# Create two small numpy arrays with distinct shapes to allow probing
A = np.array([1.0, 2.0, 3.0, 4.0])
B = np.array([5.0, 6.0, 7.0, 8.0])
ARGS = {
'A': A,
'B': B
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float64, ndim 0, dtype family float — no output shape is stated: observe_signature.py recorded float ndim 0, which disagrees with the fixture run
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| eval_comprehensiveness |
Ran ran on a synthesized input |
ink-usc/dig/metrics.py code served (permissive licence) · get_code("12923c362c99f77c") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Define a minimal forward function that returns logits
class MockModel(nn.Module):
def __init__(self, embed_dim=4, num_labels=3):
super().__init__()
self.linear = nn.Linear(embed_dim, num_labels)
def forward(self, input_embed, attention_mask=None, position_embed=None, type_embed=None, return_all_logits=False):
# input_embed shape: (batch, seq_len, embed_dim)
# Pool or use first token for classification
pooled = input_embed.mean(dim=1) # (batch, embed_dim)
logits = self.linear(pooled) # (batch, num_labels)
if return_all_logits:
return logits
return logits
forward_fn = MockModel(embed_dim=4, num_labels=3)
# Create input embeddings: (batch=1, seq_len=8, embed_dim=4)
input_embed = torch.randn(1, 8, 4)
# Create position embeddings: (batch=1, seq_len=8, embed_dim=4)
position_embed = torch.randn(1, 8, 4)
# Create type embeddings: (batch=1, seq_len=8, embed_dim=4)
type_embed = torch.randn(1, 8, 4)
# Create attention mask: (batch=1, seq_len=8)
attention_mask = torch.ones(1, 8, dtype=torch.bool)
# Create base token embeddings (not directly used in function, but provided as parameter)
base_token_emb = torch.randn(8, 4)
# Create attribution scores for each token: (seq_len=8,)
attr = torch.randn(8)
attr = torch.abs(attr) # Make positive for meaningful top-k selection
ARGS = {
"forward_fn": forward_fn,
"input_embed": input_embed,
"position_embed": position_embed,
"type_embed": type_embed,
"attention_mask": attention_mask,
"base_token_emb": base_token_emb,
"attr": attr,
"topk": 20,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| eval_log_odds |
Ran ran on a synthesized input |
ink-usc/dig/metrics.py code served (permissive licence) · get_code("ec63ecec627f3fac") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Create a small forward function that mimics a transformer-like model
class SimpleForwardFn(nn.Module):
def __init__(self, hidden_dim=4, num_classes=5):
super().__init__()
self.hidden_dim = hidden_dim
self.num_classes = num_classes
self.proj = nn.Linear(hidden_dim, num_classes)
def forward(self, input_embed, attention_mask=None, position_embed=None, type_embed=None, return_all_logits=False):
# Combine embeddings
combined = input_embed
if position_embed is not None:
combined = combined + position_embed
if type_embed is not None:
combined = combined + type_embed
# Pool or aggregate across sequence dimension
pooled = combined.mean(dim=1) # [batch, hidden_dim]
# Project to logits
logits = self.proj(pooled) # [batch, num_classes]
if return_all_logits:
return logits
return logits
forward_model = SimpleForwardFn(hidden_dim=4, num_classes=5)
forward_fn = forward_model.forward
# Create small tensors
batch_size = 1
seq_len = 6
hidden_dim = 4
num_classes = 5
input_embed = torch.randn(batch_size, seq_len, hidden_dim)
position_embed = torch.randn(batch_size, seq_len, hidden_dim)
type_embed = torch.randn(batch_size, seq_len, hidden_dim)
attention_mask = torch.ones(batch_size, seq_len, dtype=torch.float32)
base_token_emb = torch.randn(hidden_dim)
# Attribution scores for tokens (seq_len values)
attr = torch.randn(seq_len)
ARGS = {
'forward_fn': forward_fn,
'input_embed': input_embed,
'position_embed': position_embed,
'type_embed': type_embed,
'attention_mask': attention_mask,
'base_token_emb': base_token_emb,
'attr': attr,
'topk': 20
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| eval_sufficiency |
Ran ran on a synthesized input |
ink-usc/dig/metrics.py code served (permissive licence) · get_code("8e63950975ae0d89") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Build a minimal forward function that takes the required arguments
# and returns logits with shape matching what the function expects
class MockModel(nn.Module):
def __init__(self, vocab_size=10, hidden_dim=4):
super().__init__()
self.vocab_size = vocab_size
self.hidden_dim = hidden_dim
self.output_layer = nn.Linear(hidden_dim, vocab_size)
def forward(self, input_embed, attention_mask=None, position_embed=None, type_embed=None, return_all_logits=False):
# input_embed shape: (batch, seq_len, hidden_dim)
# Average pooling over sequence dimension
pooled = input_embed.mean(dim=1) # (batch, hidden_dim)
logits = self.output_layer(pooled) # (batch, vocab_size)
return logits
model = MockModel(vocab_size=10, hidden_dim=4)
def forward_fn(input_embed, attention_mask=None, position_embed=None, type_embed=None, return_all_logits=False):
"""Wrapper that calls the model"""
return model(input_embed, attention_mask=attention_mask, position_embed=position_embed, type_embed=type_embed, return_all_logits=return_all_logits)
# Create input embeddings
# Shape: (batch=1, seq_len=8, hidden_dim=4)
input_embed = torch.randn(1, 8, 4)
# Create position embeddings
# Shape: (batch=1, seq_len=8, hidden_dim=4)
position_embed = torch.randn(1, 8, 4)
# Create type embeddings
# Shape: (batch=1, seq_len=8, hidden_dim=4)
type_embed = torch.randn(1, 8, 4)
# Create attention mask
# Shape: (batch=1, seq_len=8)
attention_mask = torch.ones(1, 8, dtype=torch.bool)
# Create base token embeddings (not used in function but required parameter)
base_token_emb = torch.randn(1, 8, 4)
# Create attribution scores for each token in sequence
# Shape: (seq_len=8,)
attr = torch.randn(8)
ARGS = {
'forward_fn': forward_fn,
'input_embed': input_embed,
'position_embed': position_embed,
'type_embed': type_embed,
'attention_mask': attention_mask,
'base_token_emb': base_token_emb,
'attr': attr,
'topk': 20
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| make_monotonic_vec |
Ran ran on a synthesized input |
ink-usc/dig/monotonic_paths.py code served (permissive licence) · get_code("f8bf80d1cf1649e5") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 1, dtype family float, shape (4,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
torch.manual_seed(0)
# Analyze the function make_monotonic_vec:
# - vec1: used with >, <, ==, indexing, shape[0] -> tensor
# - vec2: used with >, <, ==, indexing, shape[0] -> tensor
# - vec3: used with >=, <=, ==, indexing, deepcopy -> tensor
# - steps: used in division (1.0 / steps) -> numeric
# Create small test tensors with shape (4,) to match the comparisons
vec1 = torch.tensor([1.0, 2.0, 3.0, 4.0])
vec2 = torch.tensor([2.0, 1.0, 3.0, 5.0])
vec3 = torch.tensor([1.5, 1.5, 3.0, 4.5])
steps = 2
ARGS = {
'vec1': vec1,
'vec2': vec2,
'vec3': vec3,
'steps': steps
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 1, dtype family float, shape (4,) — shape (4,) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| monotonic |
Ran ran on a synthesized input |
ink-usc/dig/monotonic_paths.py code served (permissive licence) · get_code("694effceefaf2cd4") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# Create three small tensors for testing the monotonic function
# The function performs element-wise comparisons and checks monotonicity constraints
vec1 = torch.tensor([1.0, 5.0, 3.0, 2.0])
vec2 = torch.tensor([2.0, 3.0, 3.0, 1.0])
vec3 = torch.tensor([1.5, 4.0, 3.0, 1.5])
ARGS = {
'vec1': vec1,
'vec2': vec2,
'vec3': vec3,
'ret': 'bool'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| predict |
Ran ran on a synthesized input |
ink-usc/dig/bert_helper.py code served (permissive licence) · get_code("f4c1f89220f52256") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 3, dtype family float, shape (2, 3, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
class SimpleTransformerModel(nn.Module):
def __init__(self, hidden_size=4, seq_length=3):
super().__init__()
self.hidden_size = hidden_size
self.seq_length = seq_length
self.linear = nn.Linear(hidden_size, hidden_size)
def forward(self, inputs_embeds, attention_mask=None):
# Model expects inputs_embeds and attention_mask
# Returns a tuple where first element is the output
output = self.linear(inputs_embeds)
return (output,)
model = SimpleTransformerModel(hidden_size=4, seq_length=3)
inputs_embeds = torch.randn(2, 3, 4) # (batch=2, seq_len=3, hidden=4)
attention_mask = torch.ones(2, 3) # (batch=2, seq_len=3)
ARGS = {
"model": model,
"inputs_embeds": inputs_embeds,
"attention_mask": attention_mask,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 3, dtype family float, shape (2, 3, 4) — shape (2, 3, 4) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| run_dig_explanation |
Ran ran on a synthesized input |
ink-usc/dig/attributions.py code served (permissive licence) · get_code("0f56782c71796ff7") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 5) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
# Build a mock DIG function object that has an attribute method
class MockDIGFunc:
def attribute(self, scaled_features, additional_forward_args, n_steps):
# Return attributions with shape (batch, seq_len, embedding_dim)
batch_size = scaled_features.shape[0]
seq_len = scaled_features.shape[1]
embed_dim = scaled_features.shape[2]
return torch.randn(batch_size, seq_len, embed_dim)
# Create small tensors with distinct dimensions
batch_size = 2
seq_len = 5
embed_dim = 4
attention_mask_size = 3
all_input_embed = torch.randn(batch_size, seq_len, embed_dim)
position_embed = torch.randn(seq_len, embed_dim)
type_embed = torch.randn(2, embed_dim) # Different size for type
attention_mask = torch.randn(batch_size, attention_mask_size)
dig_func = MockDIGFunc()
steps = 10
ARGS = {
"dig_func": dig_func,
"all_input_embed": all_input_embed,
"position_embed": position_embed,
"type_embed": type_embed,
"attention_mask": attention_mask,
"steps": steps
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 2, dtype family float, shape (2, 5) — shape (2, 5) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| summarize_attributions |
Ran ran on a synthesized input |
ink-usc/dig/attributions.py code served (permissive licence) · get_code("6fbd77010ba96ecf") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 3) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
torch.manual_seed(0)
# The function takes attributions tensor, sums over last dimension,
# squeezes dimension 0, and normalizes by Frobenius norm.
# We need a tensor with at least 3 dimensions (since we sum dim=-1 and squeeze dim=0)
attributions = torch.randn(2, 3, 4)
ARGS = {
"attributions": attributions
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type Tensor, ndim 2, dtype family float, shape (2, 3) — shape (2, 3) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| BaseExplainer |
Not yet run unverified (harvested, no recorded run) |
g8a9/ferret/ferret/explainers/gradient.py code served (permissive licence) · get_code("350432c068907cfc") How it ran |
| BaseTaskHelper |
Not yet run unverified (harvested, no recorded run) |
g8a9/ferret/ferret/explainers/gradient.py code served (permissive licence) · get_code("d15101324a574447") How it ran |
| IntegratedGradientExplainer |
Not yet run unverified (harvested, no recorded run) |
g8a9/ferret/ferret/explainers/gradient.py code served (permissive licence) · get_code("3a39393f1428fdef") How it ran |
| create_helper |
Not yet run unverified (harvested, no recorded run) |
g8a9/ferret/ferret/explainers/gradient.py code served (permissive licence) · get_code("f966d0a57a94d578") How it ran |
| nn_forward_func |
Not yet run unverified (harvested, no recorded run) |
ink-usc/dig/bert_helper.py code served (permissive licence) · get_code("74f3fd36cd31110f") How it ran |
| nn_forward_func |
Not yet run unverified (harvested, no recorded run) |
ink-usc/dig/distilbert_helper.py code served (permissive licence) · get_code("c73850601b4bfca6") How it ran |
| nn_init |
Not yet run unverified (harvested, no recorded run) |
ink-usc/dig/bert_helper.py code served (permissive licence) · get_code("1e9c64aa330afd59") How it ran |
| nn_init |
Not yet run unverified (harvested, no recorded run) |
ink-usc/dig/distilbert_helper.py code served (permissive licence) · get_code("2301db8848f0fa1f") How it ran |