SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2104.06399 · ICCV · 2021

Co-Scale Conv-Attentional Image Transformers

Zhuowen Tu, Yifan Xu, Weijian Xu, Tyler Chang

arXiv · PDF · Open in the Atlas

Code that ran

17 of 22 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (11 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 0 violated it and 16 have no contract verdict; 1 failure of our instrument; 5 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2104.06399.json · badge: /api/ran/2104.06399.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
mlpc-ucsd/CoaT marked official in Papers with Code
Papers with Code archive
7 ran · 1 no recorded run
naver-ai/vidt community
Papers with Code archive
7 ran · 1 no recorded run
rishikksh20/CoaT-pytorch community
Papers with Code archive
3 ran · 3 no recorded run
FunctionStatusWhere it lives
ConvAttention Ran
ran on a synthesized input
rishikksh20/CoaT-pytorch/coat.py
pointer only (licence: NONE) · get_code("c51e8e1183529548")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, rishikksh20/CoaT-pytorch when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

ConvPosEnc Ran
ran on a synthesized input · constructed an object; no result was computed
mlpc-ucsd/CoaT/src/models/coat.py
code served (permissive licence) · get_code("ec1951ebf6c598f7")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ConvPosEnc: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# ConvPosEnc.__init__ takes: dim, k=3
# ConvPosEnc.forward takes: x (tensor), size (tuple of H, W)

# The forward method expects:
# - x: shape (B, N, C) where N = 1 + H*W (cls_token + image tokens)
# - size: tuple (H, W)

# Let's set up small dimensions:
# B=2 (batch size)
# C=4 (channels/dim)
# H=3, W=5 (spatial dimensions, distinct for probing)
# N = 1 + 3*5 = 16

B = 2
C = 4
H = 3
W = 5
N = 1 + H * W

# Create input tensor: (B, N, C)
x = torch.randn(B, N, C)

# Create size tuple
size = (H, W)

# For __init__, we need dim and k
dim = C
k = 3

ARGS = {
    "dim": dim,
    "k": k,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ConvPosEnc — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ConvPosEnc output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ConvPosEnc Ran
ran on a synthesized input · constructed an object; no result was computed
naver-ai/vidt/methods/coat_w_ram.py
code served (permissive licence) · get_code("ca2889aa1f10dbf9")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ConvPosEnc: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# ConvPosEnc parameters:
# - dim: int (dimension of the input, used in Conv2d)
# - k: int (kernel size, default=3)

ARGS = {
    'dim': 4,
    'k': 3,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ConvPosEnc — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ConvPosEnc output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ConvRelPosEnc Ran
ran on a synthesized input · constructed an object; no result was computed
mlpc-ucsd/CoaT/src/models/coat.py
code served (permissive licence) · get_code("4e75f7053a342b72")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ConvRelPosEnc: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Parameters for ConvRelPosEnc initialization
Ch = 4  # Channels per head
h = 2   # Number of heads
window = {3: 1, 5: 1}  # Dict mapping window size to head splits

ARGS = {
    "Ch": Ch,
    "h": h,
    "window": window,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
einops, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ConvRelPosEnc — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ConvRelPosEnc output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ConvRelPosEnc Ran
ran on a synthesized input · constructed an object; no result was computed
naver-ai/vidt/methods/coat_w_ram.py
code served (permissive licence) · get_code("189e7eafbdfa6473")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ConvRelPosEnc: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Parameters for ConvRelPosEnc
Ch = 4  # Channels per head
h = 2   # Number of heads
window = {3: 1, 5: 1}  # Dict mapping window size to head splits

ARGS = {
    "Ch": Ch,
    "h": h,
    "window": window,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ConvRelPosEnc — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ConvRelPosEnc output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
FactorAtt_ConvRelPosEnc Ran
ran on a synthesized input
mlpc-ucsd/CoaT/src/models/coat.py
code served (permissive licence) · get_code("a16e3d507a921817")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

class DummyCRPE(nn.Module):
    """Dummy Convolutional Relative Position Encoding module."""
    def __init__(self, dim, num_heads):
        super().__init__()
        self.dim = dim
        self.num_heads = num_heads
        head_dim = dim // num_heads
        self.conv = nn.Conv2d(head_dim, head_dim, kernel_size=3, padding=1, groups=head_dim)
    
    def forward(self, q, v, size):
        B, num_heads, N, head_dim = q.shape
        H, W = size
        # Reshape v to spatial dimensions
        v_spatial = v.reshape(B, num_heads, H, W, head_dim).permute(0, 1, 4, 2, 3)
        v_spatial = v_spatial.reshape(B * num_heads, head_dim, H, W)
        # Apply convolution
        v_conv = self.conv(v_spatial)
        # Reshape back
        v_conv = v_conv.reshape(B, num_heads, head_dim, H, W).permute(0, 1, 3, 4, 2)
        v_conv = v_conv.reshape(B, num_heads, N, head_dim)
        return v_conv

dim = 8
num_heads = 2
batch_size = 2
H = 3
W = 4
N = H * W

shared_crpe = DummyCRPE(dim=dim, num_heads=num_heads)

INIT_ARGS = {
    'dim': dim,
    'num_heads': num_heads,
    'qkv_bias': False,
    'qk_scale': None,
    'attn_drop': 0.1,
    'proj_drop': 0.1,
    'shared_crpe': shared_crpe
}

x = torch.randn(batch_size, N, dim)
size = (H, W)

ARGS = {
    'x': x,
    'size': size
}
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 3, dtype family float — no output shape is stated: observe_signature.py observed this class with its constructor sizes canonicalised where it could, which the node does not record, so its shape is not stated as this setup block's
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
Mlp Ran
ran on a synthesized input · constructed an object; no result was computed
mlpc-ucsd/CoaT/src/models/coat.py
code served (permissive licence) · get_code("0b841fb406f1acc4")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Mlp: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    "in_features": 4,
    "hidden_features": 8,
    "out_features": 3,
    "act_layer": nn.GELU,
    "drop": 0.1,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Mlp — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Mlp output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Mlp Ran
ran on a synthesized input · constructed an object; no result was computed
naver-ai/vidt/methods/coat_w_ram.py
code served (permissive licence) · get_code("829106170239b59a")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Mlp: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    "in_features": 4,
    "hidden_features": 8,
    "out_features": 3,
    "act_layer": nn.GELU,
    "drop": 0.1,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Mlp — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Mlp output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ParallelBlock Ran
ran on a synthesized input · constructed an object; no result was computed
mlpc-ucsd/CoaT/src/models/coat.py
code served (permissive licence) · get_code("d72144e8585477b4")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ParallelBlock: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from timm.models.layers import DropPath

torch.manual_seed(0)

# Mock ConvRelPosEnc that the code expects
class MockConvRelPosEnc(nn.Module):
    def __init__(self, dim):
        super().__init__()
        self.dim = dim
    
    def forward(self, q, v, size):
        # q shape: (B, num_heads, N, head_dim)
        # v shape: (B, num_heads, N, head_dim)
        # Return same shape as q for broadcasting in attention
        B, num_heads, N, head_dim = q.shape
        return torch.zeros(B, num_heads, N, head_dim)

# Mock CPE (Convolutional Position Encoding)
class MockCPE(nn.Module):
    def __init__(self, dim):
        super().__init__()
        self.dim = dim
    
    def forward(self, x, size):
        # x shape: (B, N, C)
        # Just return x unchanged
        return x

# Build shared_crpes - list of MockConvRelPosEnc for indices 1, 2, 3
shared_crpes = [
    None,  # index 0 unused
    MockConvRelPosEnc(dim=64),  # index 1
    MockConvRelPosEnc(dim=64),  # index 2
    MockConvRelPosEnc(dim=64),  # index 3
]

# Build shared_cpes - list of MockCPE for indices 1, 2, 3
shared_cpes = [
    None,  # index 0 unused
    MockCPE(dim=64),  # index 1
    MockCPE(dim=64),  # index 2
    MockCPE(dim=64),  # index 3
]

ARGS = {
    'dims': [64, 64, 64, 64],  # All dimensions same size for the assertions
    'num_heads': 4,
    'mlp_ratios': [0.0, 4.0, 4.0, 4.0],  # Same ratio for indices 1,2,3
    'qkv_bias': False,
    'qk_scale': None,
    'drop': 0.1,
    'attn_drop': 0.1,
    'drop_path': 0.1,
    'act_layer': nn.GELU,
    'norm_layer': nn.LayerNorm,
    'shared_cpes': shared_cpes,
    'shared_crpes': shared_crpes,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
timm, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ParallelBlock — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ParallelBlock output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ParallelBlock Ran
ran on a synthesized input
rishikksh20/CoaT-pytorch/coat.py
pointer only (licence: NONE) · get_code("7b94f0de626e3564")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, rishikksh20/CoaT-pytorch when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

PatchEmbed Ran
ran on a synthesized input · constructed an object; no result was computed
mlpc-ucsd/CoaT/src/models/coat.py
code served (permissive licence) · get_code("2627bd9fa9d87539")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, PatchEmbed: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from timm.models.layers import DropPath, to_2tuple, trunc_normal_

torch.manual_seed(0)

# Create a sample input tensor for testing
# Shape: (batch_size=2, channels=3, height=32, width=32)
sample_input = torch.randn(2, 3, 32, 32)

ARGS = {
    "patch_size": 16,
    "in_chans": 3,
    "embed_dim": 768,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
timm, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type PatchEmbed — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a PatchEmbed output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
PatchEmbed Ran
ran on a synthesized input
naver-ai/vidt/methods/coat_w_ram.py
code served (permissive licence) · get_code("b418de766b04f6df")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 4, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from timm.models.layers import to_2tuple

INIT_ARGS = {
    'patch_size': 4,
    'in_chans': 3,
    'embed_dim': 8,
    'norm_layer': nn.LayerNorm
}

ARGS = {
    'x': torch.randn(2, 3, 14, 10)
}
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
timm, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 4, dtype family float — no output shape is stated: observe_signature.py observed this class with its constructor sizes canonicalised where it could, which the node does not record, so its shape is not stated as this setup block's
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
RonfiguredAttentionModule Ran
ran on a synthesized input · constructed an object; no result was computed
naver-ai/vidt/methods/coat_w_ram.py
code served (permissive licence) · get_code("9d9cd1555654bd9c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, RonfiguredAttentionModule: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Create a minimal CRPE (Convolutional Relative Position Encoding) module
class MinimalCRPE(nn.Module):
    def __init__(self, dim, num_heads):
        super().__init__()
        self.dim = dim
        self.num_heads = num_heads
        
    def forward(self, q, v, size):
        # q shape: (B, num_heads, N, head_dim)
        # v shape: (B, num_heads, N, head_dim)
        # Return shape should match attention output
        B, num_heads, N, head_dim = q.shape
        return torch.zeros(B, num_heads, N, head_dim)

# Setup parameters
dim = 4
num_heads = 2
qkv_bias = False
attn_drop = 0.0
proj_drop = 0.0
shared_crpe = MinimalCRPE(dim, num_heads)

ARGS = {
    'dim': dim,
    'num_heads': num_heads,
    'qkv_bias': qkv_bias,
    'attn_drop': attn_drop,
    'proj_drop': proj_drop,
    'shared_crpe': shared_crpe,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type RonfiguredAttentionModule — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a RonfiguredAttentionModule output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
SepConv2d Ran
ran on a synthesized input
rishikksh20/CoaT-pytorch/coat.py
pointer only (licence: NONE) · get_code("88b0cc98aec21e35")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, rishikksh20/CoaT-pytorch when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

SerialBlock Ran
ran on a synthesized input · constructed an object; no result was computed
mlpc-ucsd/CoaT/src/models/coat.py
code served (permissive licence) · get_code("9da6d64e6fdac210")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, SerialBlock: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from timm.models.layers import DropPath, to_2tuple, trunc_normal_

torch.manual_seed(0)

# Create a minimal ConvRelPosEnc that satisfies the interface
class MinimalConvRelPosEnc(nn.Module):
    def __init__(self, dim, num_heads=8):
        super().__init__()
        self.dim = dim
        self.num_heads = num_heads
    
    def forward(self, q, v, size):
        # q shape: (B, num_heads, N, head_dim)
        # v shape: (B, num_heads, N, head_dim)
        # Should return tensor of shape (B, num_heads, N, head_dim)
        B, H, N, D = q.shape
        return torch.zeros(B, H, N, D)

# Create a minimal ConvPosEnc that satisfies the interface
class MinimalConvPosEnc(nn.Module):
    def __init__(self, dim):
        super().__init__()
        self.dim = dim
    
    def forward(self, x, size):
        # x shape: (B, N, C)
        # Should return same shape
        return x

# Construct arguments for SerialBlock
dim = 4
num_heads = 2
mlp_ratio = 4.0
qkv_bias = False
qk_scale = None
drop = 0.1
attn_drop = 0.1
drop_path = 0.1
act_layer = nn.GELU
norm_layer = nn.LayerNorm

shared_cpe = MinimalConvPosEnc(dim)
shared_crpe = MinimalConvRelPosEnc(dim, num_heads=num_heads)

ARGS = {
    'dim': dim,
    'num_heads': num_heads,
    'mlp_ratio': mlp_ratio,
    'qkv_bias': qkv_bias,
    'qk_scale': qk_scale,
    'drop': drop,
    'attn_drop': attn_drop,
    'drop_path': drop_path,
    'act_layer': act_layer,
    'norm_layer': norm_layer,
    'shared_cpe': shared_cpe,
    'shared_crpe': shared_crpe,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
timm, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type SerialBlock — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a SerialBlock output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
SerialBlock Ran
ran on a synthesized input · constructed an object; no result was computed
naver-ai/vidt/methods/coat_w_ram.py
code served (permissive licence) · get_code("9cde5c1a6de701e6")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, SerialBlock: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from timm.models.layers import DropPath, to_2tuple, trunc_normal_

torch.manual_seed(0)

# Mock CRPE module that implements the interface used in RonfiguredAttentionModule
class MockCRPE(nn.Module):
    def __init__(self, dim, num_heads):
        super().__init__()
        self.dim = dim
        self.num_heads = num_heads
    
    def forward(self, q, v, size):
        # q shape: (B, num_heads, N, head_dim)
        # v shape: (B, num_heads, N, head_dim)
        # Returns tensor of same shape as attention output
        B, num_heads, N, head_dim = q.shape
        return torch.zeros(B, num_heads, N, head_dim)

# Mock CPE module for positional encoding
class MockCPE(nn.Module):
    def __init__(self, dim):
        super().__init__()
        self.dim = dim
    
    def forward(self, x, size):
        return x

# Build shared modules
dim = 4
num_heads = 2
shared_crpe = MockCRPE(dim, num_heads)
shared_cpe = MockCPE(dim)

# Create SerialBlock instance with all required parameters
serial_block = nn.Sequential()  # placeholder - will be replaced

ARGS = {
    'dim': 4,
    'num_heads': 2,
    'mlp_ratio': 4.0,
    'qkv_bias': False,
    'drop': 0.1,
    'attn_drop': 0.0,
    'drop_path': 0.0,
    'act_layer': nn.GELU,
    'norm_layer': nn.LayerNorm,
    'shared_cpe': shared_cpe,
    'shared_crpe': shared_crpe,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
timm, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type SerialBlock — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a SerialBlock output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
masked_sin_pos_encoding Ran
ran; our contract draft was wrong, not the code
naver-ai/vidt/methods/coat_w_ram.py
code served (permissive licence) · get_code("a6ff9224dd83c873")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 4, dtype family float, shape (2, 4, 3, 8) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import math

torch.manual_seed(0)

# Construct arguments for masked_sin_pos_encoding
# x: [PATCH] tokens - shape should be [batch, height, width] based on cumsum operations in mask
# mask: padding mask for [PATCH] tokens - boolean tensor with same spatial dims as x
# num_pos_feats: channel dimension size
# temperature: temperature value (default 10000)
# scale: normalization scale (default 2 * math.pi)

batch_size = 2
height = 4
width = 3
num_pos_feats = 8

# x: token tensor [batch, height, width]
x = torch.randn(batch_size, height, width)

# mask: padding mask [batch, height, width] - True for padded positions, False for valid
mask = torch.tensor([
    [[False, False, True], [False, False, True], [False, False, True], [False, False, True]],
    [[False, False, False], [False, False, False], [True, True, True], [True, True, True]]
])

temperature = 10000
scale = 2 * math.pi

ARGS = {
    'x': x,
    'mask': mask,
    'num_pos_feats': num_pos_feats,
    'temperature': temperature,
    'scale': scale
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 4, dtype family float, shape (2, 4, 3, 8) — shape (2, 4, 3, 8) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
CoaT Not yet run
unverified (harvested, no recorded run)
mlpc-ucsd/CoaT/src/models/coat.py
code served (permissive licence) · get_code("13863a3db32a6073")
How it ran

no invocation recorded.

CoaT Not yet run
unverified (harvested, no recorded run)
naver-ai/vidt/methods/coat_w_ram.py
code served (permissive licence) · get_code("9a1d68943ad59aa4")
How it ran

no invocation recorded.

CoaT Not yet run
unverified (harvested, no recorded run)
rishikksh20/CoaT-pytorch/coat.py
pointer only (licence: NONE) · get_code("0d4a68dc6be44b68")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, rishikksh20/CoaT-pytorch when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

SerialBlock Not yet run
unverified (harvested, no recorded run)
rishikksh20/CoaT-pytorch/coat.py
pointer only (licence: NONE) · get_code("9e0aeb978b4f9dd5")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, rishikksh20/CoaT-pytorch when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

Transformer Not yet run
unverified (harvested, no recorded run)
rishikksh20/CoaT-pytorch/coat.py
pointer only (licence: NONE) · get_code("34b5b23322f680bd")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, rishikksh20/CoaT-pytorch when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

Also present in other papers' code

Of 22 samples lifted from this paper's linked repositories, 22 carry a function identity and 0 do not. Of the 21 distinct functions with an identity, 2 are also present in other papers whose code we hold (2 with at least one paper dated earlier, 1 the same month, 2 later) and 19 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
Mlp
brick of this paper's assembly: model
130 other papers
12 dated earlier, 1 the same month, 117 later · 114 only as an assembly brick
2003.12039 (2020-03)
RAFT: Recurrent All-Pairs Field Transforms for Optical Flow
2010.11929 (2020-10)
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
2012.12877 (2020-12)
Training data-efficient image transformers & distillation through attention
and 127 more not named here
PatchEmbed
brick of this paper's assembly: model
6 other papers
1 dated earlier, 5 later · 3 only as an assembly brick
2103.14030 (2021-03)
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
2111.09886 (2021-11)
SimMIM: a Simple Framework for Masked Image Modeling
2208.03792 (2022-08)
Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objects
and 3 more not named here

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

In this paper, we present Co-scale conv-attentional image Transformers (CoaT), a Transformer-based image classifier equipped with co-scale and conv-attentional mechanisms. First, the co-scale mechanism maintains the integrity of Transformers' encoder branches at individual scales, while allowing representations learned at different scales to effectively communicate with each other; we design a series of serial and parallel blocks to realize the co-scale mechanism. Second, we devise a conv-attentional mechanism by realizing a relative position embedding formulation in the factorized attention module with an efficient convolution-like implementation. CoaT empowers image Transformers with enriched multi-scale and contextual modeling capabilities. On ImageNet, relatively small CoaT models attain superior classification results compared with similar-sized convolutional neural networks and image/vision Transformers. The effectiveness of CoaT's backbone is also illustrated on object detection and instance segmentation, demonstrating its applicability to downstream computer vision tasks. * indicates equal contribution. Code at https://github.com/mlpc-ucsd/CoaT.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2104.06399")
get_code_for_paper("2104.06399")
have("2104.06399")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2104.06399.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2104.06399.svg)](https://syntology.ai/paper/2104.06399)

Connect an agent — have() is free.