SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2103.12731 · CVPR · 2021

Scaling Local Self-Attention for Parameter Efficient Visual Backbones

U Berkeley, Aravind Srinivas, Jonathon Shlens, Ashish Vaswani, Blake Hechtman, Niki Parmar

arXiv · PDF · Open in the Atlas

Code that ran

18 of 20 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (4 of them constructed an object rather than computing a result; 1 honoured the contract we drafted, 0 violated it and 10 have no contract verdict; 7 failures of our instrument; 2 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2103.12731.json · badge: /api/ran/2103.12731.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
kevin-ssy/ViP community (classifier label)
classifier basis not recorded on the link
6 ran · 2 no recorded run
xmu-xiaoma666/External-Attention-pytorch community
Papers with Code archive
6 ran · 0 no recorded run
qwopqwop200/halonet-tf2 community
Papers with Code archive
5 ran · 0 no recorded run
lucidrains/halonet-pytorch community (classifier label)
classifier basis not recorded on the link
1 ran · 0 no recorded run
FunctionStatusWhere it lives
HaloAttention Ran
ran on a synthesized input · constructed an object; no result was computed
xmu-xiaoma666/External-Attention-pytorch/model/attention/HaloAttention.py
code served (permissive licence) · get_code("2e1c36d6346e85e2")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, HaloAttention: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn

torch.manual_seed(0)

ARGS = {
    'dim': 16,
    'block_size': 2,
    'halo_size': 1,
    'dim_head': 4,
    'heads': 2,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
einops, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type HaloAttention — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a HaloAttention output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
HaloAttention Ran
ran on a synthesized input · constructed an object; no result was computed
qwopqwop200/halonet-tf2/halonet.py
code served (permissive licence) · get_code("e672b9aa56a1370e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, HaloAttention: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import tensorflow as tf
import numpy as np

# Set random seed for reproducibility
tf.random.set_seed(0)
np.random.seed(0)

# Create a small input tensor
# Shape: (batch_size, height, width, channels)
# Using distinct dimensions: batch=2, height=8, width=8, channels=4
x = tf.random.normal((2, 8, 8, 4), dtype=tf.float32)

# Define ARGS dictionary with all parameters for HaloAttention
ARGS = {
    'block_size': 4,      # Block size for dividing the spatial dimensions
    'halo_size': 1,       # Halo size for expanding the attention window
    'dim_head': 2,        # Dimension per head (kept small for testing)
    'heads': 2            # Number of attention heads
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
einops, numpy, tensorflow — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type HaloAttention — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a HaloAttention output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
RelPosEmb Ran
ran on a synthesized input · constructed an object; no result was computed
xmu-xiaoma666/External-Attention-pytorch/model/attention/HaloAttention.py
code served (permissive licence) · get_code("84902fc5aa98fdd2")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, RelPosEmb: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn

torch.manual_seed(0)

# Analyze the RelPosEmb.__init__ parameters:
# - block_size: int, used in forward() to reshape tensors
# - rel_size: int, used to create rel_height and rel_width parameters with shape (rel_size*2-1, dim_head)
# - dim_head: int, used as the second dimension of parameter tensors

# From the forward method, we can see:
# - q is reshaped as 'b (x y) c -> b x y c' with x=block_size
# - This means the input has shape (b, block_size*block_size, dim_head)
# - rel_height and rel_width have shapes (rel_size*2-1, dim_head)
# - The forward pass processes q through relative_logits_1d

# Choose small distinct values for dimensions
block_size = 4
rel_size = 3
dim_head = 8

ARGS = {
    'block_size': block_size,
    'rel_size': rel_size,
    'dim_head': dim_head,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
einops, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type RelPosEmb — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a RelPosEmb output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
RelPosEmb Ran
ran on a synthesized input · constructed an object; no result was computed
qwopqwop200/halonet-tf2/halonet.py
code served (permissive licence) · get_code("f84f90d761d35986")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, RelPosEmb: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import tensorflow as tf
import numpy as np

# Set seed for reproducibility
tf.random.set_seed(0)
np.random.seed(0)

# Define ARGS for RelPosEmb.__init__
ARGS = {
    'block_size': 4,
    'rel_size': 3,
    'dim_head': 8
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
einops, numpy, tensorflow — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type RelPosEmb — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a RelPosEmb output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
addmm_flop_jit Ran
ran on a synthesized input
kevin-ssy/ViP/utils/flop_count/jit_handles.py
code served (permissive licence) · get_code("27b31e965ef896f1")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Counter. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from collections import Counter
from unittest.mock import Mock

torch.manual_seed(0)

# Create mock jit Value objects that behave like torch._C.Value
class MockJitValue:
    def __init__(self, shape):
        self._shape = shape
    
    def isCompleteTensor(self):
        return True
    
    def type(self):
        return Mock(sizes=lambda: self._shape)

# Create inputs and outputs lists
# addmm_flop_jit expects:
# inputs[1:3] to be two 2D tensors (it skips inputs[0])
# The function extracts shapes from inputs[1] and inputs[2]

# inputs[1] should have shape [batch_size, input_dim]
# inputs[2] should have shape [input_dim, output_dim]
mock_input_1 = MockJitValue([2, 4])  # batch_size=2, input_dim=4
mock_input_2 = MockJitValue([4, 3])  # input_dim=4, output_dim=3
mock_input_0 = MockJitValue([3])     # bias term (not used in shape extraction)

inputs = [mock_input_0, mock_input_1, mock_input_2]
outputs = []  # outputs parameter is not used in the function

ARGS = {
    'inputs': inputs,
    'outputs': outputs
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Counter — a Counter output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a Counter output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
apply_pos Ran
ran on a synthesized input
kevin-ssy/ViP/models/vip_layers.py
code served (permissive licence) · get_code("24b72f9389954e2e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 3, dtype family float, shape (2, 3, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from einops import rearrange

torch.manual_seed(0)

# Create small test tensors
# tensor: shape (batch, n, heads*channels) - will be rearranged to (batch, n, heads, channels)
batch_size = 2
n = 3
num_heads = 2
channels = 2
total_dim = num_heads * channels

tensor = torch.randn(batch_size, n, total_dim)

# pos: shape (batch, n, heads, channels) - for broadcasting after rearrange
pos = torch.randn(batch_size, n, num_heads, channels)

num_heads = num_heads

ARGS = {
    "tensor": tensor,
    "pos": pos,
    "num_heads": num_heads
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
einops, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 3, dtype family float, shape (2, 3, 4) — shape (2, 3, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
create_loader Ran
ran on a synthesized input
kevin-ssy/ViP/utils/loader.py
code served (permissive licence) · get_code("1cf1c5e0f9a581ba")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned PrefetchLoader. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.utils.data
from torch.utils.data import Dataset

torch.manual_seed(0)

class DummyDataset(Dataset):
    def __init__(self, num_samples=4):
        self.num_samples = num_samples
        self.transform = None
    
    def __len__(self):
        return self.num_samples
    
    def __getitem__(self, idx):
        # Return (image, label) tuple
        image = torch.randn(3, 224, 224)
        label = torch.tensor(idx % 10, dtype=torch.long)
        if self.transform is not None:
            image = self.transform(image)
        return image, label

dataset = DummyDataset(num_samples=8)

ARGS = {
    'dataset': dataset,
    'input_size': 224,
    'batch_size': 2,
    'is_training': False,
    'use_prefetcher': True,
    'no_aug': False,
    're_prob': 0.0,
    're_mode': 'const',
    're_count': 1,
    're_split': False,
    'scale': None,
    'ratio': None,
    'hflip': 0.5,
    'vflip': 0.0,
    'color_jitter': 0.4,
    'auto_augment': None,
    'num_aug_splits': 0,
    'interpolation': 'bilinear',
    'mean': (0.485, 0.456, 0.406),
    'std': (0.229, 0.224, 0.225),
    'num_workers': 0,
    'distributed': False,
    'crop_pct': None,
    'collate_fn': None,
    'pin_memory': False,
    'fp16': False,
    'tf_preprocessing': False,
    'use_multi_epochs_loader': False,
    'repeated_aug': False,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
timm, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type PrefetchLoader — a PrefetchLoader output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a PrefetchLoader output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
expand_dim Ran
ran; our contract draft was wrong, not the code
xmu-xiaoma666/External-Attention-pytorch/model/attention/HaloAttention.py
code served (permissive licence) · get_code("042da966a79899bf")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 3, dtype family float, shape (2, 3, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create a small tensor with distinct dimensions: batch=2, features=4
t = torch.randn(2, 4)

# dim parameter: which dimension to expand
dim = 1

# k parameter: how many times to expand that dimension
k = 3

ARGS = {
    "t": t,
    "dim": dim,
    "k": k,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 3, dtype family float, shape (2, 3, 4) — shape (2, 3, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
expand_dim Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
qwopqwop200/halonet-tf2/halonet.py
code served (permissive licence) · get_code("c408fb0e411bf284")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned EagerTensor, ndim 3, dtype family float, shape (2, 3, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import tensorflow as tf

# Create a small test tensor
# Using shape (2, 4) - batch size 2, 4 features
t = tf.constant([[1.0, 2.0, 3.0, 4.0],
                  [5.0, 6.0, 7.0, 8.0]])

# dim: which axis to expand and tile along
dim = 1

# k: how many times to tile along that dimension
k = 3

ARGS = {
    't': t,
    'dim': dim,
    'k': k
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
tensorflow — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type EagerTensor, ndim 3, dtype family float, shape (2, 3, 4) — shape (2, 3, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
generic_activation_jit Ran
ran on a synthesized input
kevin-ssy/ViP/utils/flop_count/jit_handles.py
code served (permissive licence) · get_code("29724655612e3ef9")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned function. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
from collections import Counter
from typing import List, Any

torch.manual_seed(0)

# Create a mock JIT Value object that has the methods called in the function
class MockJITValue:
    def __init__(self, shape):
        self._shape = shape
        self._is_complete = True
    
    def isCompleteTensor(self):
        return self._is_complete
    
    def type(self):
        return self
    
    def sizes(self):
        return self._shape

# Test the function with a mock output list
mock_outputs = [MockJITValue([2, 3, 4, 5])]

ARGS = {
    "op_name": "conv2d"
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type function — a function output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a function output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_shape Ran
ran on a synthesized input
kevin-ssy/ViP/utils/flop_count/jit_handles.py
code served (permissive licence) · get_code("653ac611901cfc1c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.jit

torch.manual_seed(0)

# Create a simple TorchScript module to get a jit Value object
class SimpleModule(torch.nn.Module):
    def forward(self, x):
        return x + 1

# Trace the module to get a jit graph
module = SimpleModule()
example_input = torch.randn(2, 4)
traced = torch.jit.trace(module, example_input)

# Extract a jit Value object from the graph
# We need to get a Value that represents a complete tensor
graph = traced.graph
for node in graph.nodes():
    for output in node.outputs():
        val = output
        break

ARGS = {
    "val": val
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family int — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
load_ext Ran
ran on a synthesized input
kevin-ssy/ViP/utils/ext_loader.py
code served (permissive licence) · get_code("a512212dfdd82da0")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned module. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import importlib
import sys
from types import ModuleType

# Create a mock module with the required functions
mock_module = ModuleType('mock_ext')
mock_module.func1 = lambda: "function1"
mock_module.func2 = lambda: "function2"
mock_module.func3 = lambda: "function3"

# Register the mock module so it can be imported
sys.modules['mock_ext'] = mock_module

ARGS = {
    'name': 'mock_ext',
    'funcs': ['func1', 'func2', 'func3']
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type module — a module output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a module output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
pair Ran
ran, honoured the contract we drafted
lucidrains/halonet-pytorch/halonet_pytorch/halonet_pytorch.py
code served (permissive licence) · get_code("3e27684df29ad41d")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned tuple, ndim 3, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# The function `pair` takes a single parameter `x` and returns either (x, x) if x is not a tuple,
# or returns x itself if it is already a tuple.
# We need to test both cases.

# Test case 1: x is not a tuple (e.g., a tensor, int, or other object)
x_not_tuple = torch.randn(2, 4)

ARGS = {
    "x": x_not_tuple
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 3, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
rel_to_abs Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
xmu-xiaoma666/External-Attention-pytorch/model/attention/HaloAttention.py
code served (permissive licence) · get_code("1e87e34e9818520d")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 3, dtype family float, shape (2, 3, 2) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from einops import rearrange, repeat

torch.manual_seed(0)

# The function rel_to_abs takes a single parameter 'x' which is a tensor
# It uses x.shape to get (b, l, m)
# It calls to(x) which accesses x.device and x.dtype
# It performs tensor operations on x

# Create a small test tensor with distinct dimensions
# b=2 (batch), l=3 (length), m=4 (some dimension)
x = torch.randn(2, 3, 4, dtype=torch.float32)

ARGS = {
    'x': x
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
einops, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 3, dtype family float, shape (2, 3, 2) — shape (2, 3, 2) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
rel_to_abs Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
qwopqwop200/halonet-tf2/halonet.py
code served (permissive licence) · get_code("de5b53fe2be91d13")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned EagerTensor, ndim 3, dtype family float, shape (2, 4, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import tensorflow as tf
from einops import rearrange, repeat

# Create a small test tensor for the rel_to_abs function
# The function expects input shape (b, l, m) where:
# b = batch size
# l = sequence length
# m = some dimension
b, l, m = 2, 4, 7

x = tf.random.normal((b, l, m))

ARGS = {
    'x': x
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
einops, tensorflow — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type EagerTensor, ndim 3, dtype family float, shape (2, 4, 4) — shape (2, 4, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
relative_logits_1d Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
xmu-xiaoma666/External-Attention-pytorch/model/attention/HaloAttention.py
code served (permissive licence) · get_code("f0386252d04dce23")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 5, dtype family float, shape (2, 3, 4, 4, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn

torch.manual_seed(0)

# Analyze the function:
# - relative_logits_1d takes: q (tensor), rel_k (tensor)
# - q is used as: shape (b, h, w, _), einsum with rel_k
# - rel_k is used as: shape[0], einsum with q
# - The function performs einsum('b x y d, r d -> b x y r', q, rel_k)
# - This means q has shape (b, x, y, d) and rel_k has shape (r, d)

# Build small, distinct tensors:
# b=2 (batch), x=3 (height), y=4 (width), d=5 (feature dim), r=7 (relation dim)
b, h, w, d = 2, 3, 4, 5
r = 7

# q shape: (b, h, w, d) = (2, 3, 4, 5)
q = torch.randn(b, h, w, d)

# rel_k shape: (r, d) = (7, 5)
# The function expects rel_k.shape[0] to be used for (r_dim + 1) // 2 calculation
rel_k = torch.randn(r, d)

ARGS = {
    'q': q,
    'rel_k': rel_k
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
einops, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 5, dtype family float, shape (2, 3, 4, 4, 4) — shape (2, 3, 4, 4, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
relative_logits_1d Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
qwopqwop200/halonet-tf2/halonet.py
code served (permissive licence) · get_code("d72140427ae8af3b")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned EagerTensor, ndim 5, dtype family float, shape (2, 3, 3, 4, 3) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import tensorflow as tf
import numpy as np
from einops import rearrange, repeat

# Set seed for reproducibility
tf.random.set_seed(0)

# Create small tensor arguments
# q: shape (b, h, w, d) where b=batch, h=height, w=width, d=depth
# From the function: b, h, w, _ = q.shape
# Used in einsum: 'b x y d, r d -> b x y r'
q = tf.constant(np.random.randn(2, 3, 4, 8).astype(np.float32))

# rel_k: shape (r, d) where r is relative position dimension
# From the function: r = (rel_k.shape[0] + 1) // 2
# Used in einsum: 'b x y d, r d -> b x y r'
# Choose r such that it's reasonable; let's use rel_k with shape (5, 8)
# This makes r = (5 + 1) // 2 = 3
rel_k = tf.constant(np.random.randn(5, 8).astype(np.float32))

ARGS = {
    "q": q,
    "rel_k": rel_k
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
einops, numpy, tensorflow — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type EagerTensor, ndim 5, dtype family float, shape (2, 3, 3, 4, 3) — shape (2, 3, 3, 4, 3) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
to Ran
ran; our contract draft was wrong, not the code
xmu-xiaoma666/External-Attention-pytorch/model/attention/HaloAttention.py
code served (permissive licence) · get_code("0bb26a83df4e7b0b")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create a small tensor to pass to the function
x = torch.randn(2, 4, dtype=torch.float32)

ARGS = {
    'x': x
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
dist_init Not yet run
unverified (harvested, no recorded run)
kevin-ssy/ViP/utils/distributed_utils.py
code served (permissive licence) · get_code("b2ada9fb7de2d355")
How it ran

no invocation recorded.

init_device Not yet run
unverified (harvested, no recorded run)
kevin-ssy/ViP/utils/distributed_utils.py
code served (permissive licence) · get_code("5d8cf2436bdfc77b")
How it ran

no invocation recorded.

Also present in other papers' code

Of 20 samples lifted from this paper's linked repositories, 20 carry a function identity and 0 do not. Of the 20 distinct functions with an identity, 8 are also present in other papers whose code we hold (2 with at least one paper dated earlier, 0 the same month, 8 later) and 12 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
expand_dim
brick of this paper's assembly: helper
4 other papers
1 dated earlier, 3 later
2101.11605 (2021-01)
Bottleneck Transformers for Visual Recognition
2310.11881 (2023-10)
A Comparative Study of Image Restoration Networks for General Backbone Network Design
2404.07846 (2024-04)
Rethinking Transformer-Based Blind-Spot Network for Self-Supervised Image Denoising
and 1 more not named here
pair 3 other papers
1 dated earlier, 2 later
2101.11605 (2021-01)
Bottleneck Transformers for Visual Recognition
2310.11881 (2023-10)
A Comparative Study of Image Restoration Networks for General Backbone Network Design
2404.07846 (2024-04)
Rethinking Transformer-Based Blind-Spot Network for Self-Supervised Image Denoising
to
brick of this paper's assembly: helper
2 other papers
2 later
2310.11881 (2023-10)
A Comparative Study of Image Restoration Networks for General Backbone Network Design
2404.07846 (2024-04)
Rethinking Transformer-Based Blind-Spot Network for Self-Supervised Image Denoising
addmm_flop_jit 1 other paper
1 later
2108.01684 (2021-08)
Vision Transformer with Progressive Sampling
create_loader 1 other paper
1 later
2108.01684 (2021-08)
Vision Transformer with Progressive Sampling
dist_init 1 other paper
1 later
2108.01684 (2021-08)
Vision Transformer with Progressive Sampling
generic_activation_jit 1 other paper
1 later
2108.01684 (2021-08)
Vision Transformer with Progressive Sampling
get_shape 1 other paper
1 later
2108.01684 (2021-08)
Vision Transformer with Progressive Sampling

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Self-attention has the promise of improving computer vision systems due to parameter-independent scaling of receptive fields and content-dependent interactions, in contrast to parameter-dependent scaling and content-independent interactions of convolutions. Self-attention models have recently been shown to have encouraging improvements on accuracyparameter trade-offs compared to baseline convolutional models such as ResNet-50. In this work, we develop selfattention models that can outperform not just the canonical baseline models, but even the high-performing convolutional models. We propose two extensions to self-attention that, in conjunction with a more efficient implementation of selfattention, improve the speed, memory usage, and accuracy of these models. We leverage these improvements to develop a new self-attention model family, HaloNets, which reach state-of-the-art accuracies on the parameter-limited setting of the ImageNet classification benchmark. In preliminary transfer learning experiments, we find that HaloNet models outperform much larger models and have better inference performance. On harder tasks such as object detection and instance segmentation, our simple local self-attention and convolutional hybrids show improvements over very strong baselines. These results mark another step in demonstrating the efficacy of self-attention models on settings traditionally dominated by convolutions. 1

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2103.12731")
get_code_for_paper("2103.12731")
have("2103.12731")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2103.12731.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2103.12731.svg)](https://syntology.ai/paper/2103.12731)

Connect an agent — have() is free.