SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2207.06405 · NeurIPS · 2022

Masked Autoencoders that Listen

Po-Yao Huang, Florian Metze, Hu Xu, Juncheng Li, Alexei Baevski, Michael Auli, Wojciech Galuba, Christoph Feichtenhofer

arXiv · PDF · Open in the Atlas

Code that ran

17 of 32 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (12 of them constructed an object rather than computing a result; 1 honoured the contract we drafted, 1 violated it and 14 have no contract verdict; 1 failure of our instrument; 15 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2207.06405.json · badge: /api/ran/2207.06405.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
rishikksh20/AudioMAE-pytorch reimplementation (classifier label)
classifier basis: classifier_2025
9 ran · 2 no recorded run
eml-eda/tle-supervised community
Papers with Code archive
5 ran · 1 no recorded run
facebookresearch/multimodal community
Papers with Code archive
2 ran · 9 no recorded run
facebookresearch/audiomae community
Papers with Code archive
1 ran · 3 no recorded run
FunctionStatusWhere it lives
Attention Ran
ran on a synthesized input · constructed an object; no result was computed
rishikksh20/AudioMAE-pytorch/audio_mae.py
code served (permissive licence) · get_code("3bd6d4930cb3f2f8")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Attention: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from einops import rearrange
from torch import einsum

torch.manual_seed(0)

# The Attention module expects:
# - dim: input dimension
# - heads: number of attention heads (default 8)
# - dim_head: dimension per head (default 64)
# - dropout: dropout rate (default 0.0)

ARGS = {
    "dim": 64,
    "heads": 4,
    "dim_head": 16,
    "dropout": 0.1,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
einops, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Attention — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Attention output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
CyclicShift Ran
ran on a synthesized input · constructed an object; no result was computed
rishikksh20/AudioMAE-pytorch/audio_mae.py
code served (permissive licence) · get_code("b20098ca471958b8")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, CyclicShift: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn

torch.manual_seed(0)

class CyclicShift(nn.Module):

    def __init__(self, displacement):
        super().__init__()
        self.displacement = displacement

    def forward(self, x):
        return torch.roll(x, shifts=(self.displacement, self.displacement), dims=(1, 2))

# Create setup for CyclicShift
# The function takes a displacement parameter which is an integer
# It's used in torch.roll with shifts parameter, so it should be an int

ARGS = {
    'displacement': 2
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type CyclicShift — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a CyclicShift output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
FeedForward Ran
ran on a synthesized input · constructed an object; no result was computed
eml-eda/tle-supervised/models/models_audio_mae.py
pointer only (licence: NONE) · get_code("a78cda64d88bd1f2")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, eml-eda/tle-supervised when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

MLP Ran
ran on a synthesized input
facebookresearch/multimodal/torchmultimodal/models/masked_auto_encoder/model.py
code served (permissive licence) · get_code("084f9fe01c1b3cf0")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 2, dtype family float, shape (2, 3) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn

INIT_ARGS = {
    "in_dim": 4,
    "out_dim": 3,
    "hidden_dims": [8, 6],
    "dropout": 0.5,
    "activation": nn.ReLU,
    "normalization": nn.BatchNorm1d,
}

ARGS = {
    "x": torch.randn(2, 4),
}

mlp = nn.Module()
mlp.__class__ = type('MLP', (nn.Module,), {
    '__init__': lambda self, in_dim, out_dim, hidden_dims=None, dropout=0.5, activation=nn.ReLU, normalization=None: (
        nn.Module.__init__(self),
        self._init_mlp(in_dim, out_dim, hidden_dims, dropout, activation, normalization)
    ),
    '_init_mlp': lambda self, in_dim, out_dim, hidden_dims, dropout, activation, normalization: (
        setattr(self, 'model', self._build_model(in_dim, out_dim, hidden_dims, dropout, activation, normalization))
    ),
    '_build_model': lambda self, in_dim, out_dim, hidden_dims, dropout, activation, normalization: (
        nn.Sequential(*self._build_layers(in_dim, out_dim, hidden_dims, dropout, activation, normalization))
    ),
    '_build_layers': lambda self, in_dim, out_dim, hidden_dims, dropout, activation, normalization: (
        self._get_layers(in_dim, out_dim, hidden_dims, dropout, activation, normalization)
    ),
    '_get_layers': lambda self, in_dim, out_dim, hidden_dims, dropout, activation, normalization: (
        (lambda layers: layers)([
            layer for hidden_dim in (hidden_dims if hidden_dims else [])
            for layer in (
                [nn.Linear(in_dim if i == 0 else hidden_dims[i-1], hidden_dim)] +
                ([normalization(hidden_dim)] if normalization else []) +
                [activation()] +
                ([nn.Dropout(dropout)] if dropout > 0 else [])
            )
            for i in [0]
        ] + [nn.Linear(hidden_dims[-1] if hidden_dims else in_dim, out_dim)])
    ),
    'forward': lambda self, x: self.model(x),
})

from typing import Callable, List, Optional, Union

class MLP(nn.Module):
    def __init__(self, in_dim: int, out_dim: int, hidden_dims: Optional[Union[int, List[int]]]=None, dropout: float=0.5, activation: Callable[..., nn.Module]=nn.ReLU, normalization: Optional[Callable[..., nn.Module]]=None) -> None:
        super().__init__()
        layers = nn.ModuleList()
        if hidden_dims is None:
            hidden_dims = []
        if isinstance(hidden_dims, int):
            hidden_dims = [hidden_dims]
        current_in_dim = in_dim
        for hidden_dim in hidden_dims:
            layers.append(nn.Linear(current_in_dim, hidden_dim))
            if normalization:
                layers.append(normalization(hidden_dim))
            layers.append(activation())
            if dropout > 0:
                layers.append(nn.Dropout(dropout))
            current_in_dim = hidden_dim
        layers.append(nn.Linear(current_in_dim, out_dim))
        self.model = nn.Sequential(*layers)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        return self.model(x)

mlp_instance = MLP(**INIT_ARGS)
output = mlp_instance(**ARGS)
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 3) — shape (2, 3) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
MultiHeadSelfAttention Ran
ran on a synthesized input
facebookresearch/multimodal/torchmultimodal/models/masked_auto_encoder/model.py
code served (permissive licence) · get_code("354dccecc7d137d8")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float, shape (2, 3, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn, Tensor
import torch.nn.functional as F

from typing import Optional

class MultiHeadSelfAttention(nn.Module):
    """
    Multihead self attention.
    Similar to the self attention variant of MHA in attention.py but uses the scaled_dot_product_attention from PyTorch
    (which uses flash or memory efficient version for certain conditions).
    TODO: merge this into attention.py once other models are ready to use it.

    Args:
        embed_dim (int): embedding dimension of the input
        num_heads (int): number of attn heads
        dropout (float): dropout rate
    """

    def __init__(self, embed_dim: int, num_heads: int, dropout: float=0.0):
        super().__init__()
        self.input_proj = nn.Linear(embed_dim, 3 * embed_dim)
        self.output_proj = nn.Linear(embed_dim, embed_dim)
        self.num_heads = num_heads
        self.dropout = dropout

    def forward(self, query: Tensor, attn_mask: Optional[Tensor]=None, is_causal: bool=False) -> Tensor:
        """
        Args:
            query (Tensor): input query of shape bsz x seq_len x embed_dim
            attn_mask (optional Tensor): attention mask of shape bsz x num_heads x seq_len x seq_len.
            Note that the num_heads dimension can equal 1 and the mask will be broadcasted to all heads.
            Two types of masks are supported.
            A boolean mask where a value of True indicates that the element should take part in attention.
            A float mask of the same type as query that is added to the attention score.
            is_causal (bool): If true, does causal attention masking. attn_mask should be set to None if this is set to True

        Returns:
            attention output Tensor of shape bsz x seq_len x embed_dim
        """
        bsz = query.size(0)
        embed_dim = query.size(-1)
        projected_query = self.input_proj(query)
        query, key, value = projected_query.chunk(3, dim=-1)
        head_dim = embed_dim // self.num_heads
        query = query.view(bsz, -1, self.num_heads, head_dim).transpose(1, 2)
        key = key.view(bsz, -1, self.num_heads, head_dim).transpose(1, 2)
        value = value.view(bsz, -1, self.num_heads, head_dim).transpose(1, 2)
        dropout = self.dropout if self.training else 0.0
        attn = F.scaled_dot_product_attention(query, key, value, attn_mask, dropout, is_causal)
        attn = attn.transpose(1, 2).reshape(bsz, -1, embed_dim)
        attn_out = self.output_proj(attn)
        return attn_out


INIT_ARGS = {
    "embed_dim": 4,
    "num_heads": 2,
    "dropout": 0.1,
}

ARGS = {
    "query": torch.randn(2, 3, 4),
    "attn_mask": None,
    "is_causal": False,
}

if __name__ == "__main__":
    model = MultiHeadSelfAttention(**INIT_ARGS)
    model.eval()
    output = model(**ARGS)
    print(f"Output shape: {output.shape}")
    print(f"Expected shape: (2, 3, 4)")
    assert output.shape == (2, 3, 4), f"Shape mismatch: {output.shape}"
    print("Test passed!")
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 3, dtype family float, shape (2, 3, 4) — shape (2, 3, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
PatchEmbed Ran
ran on a synthesized input · constructed an object; no result was computed
rishikksh20/AudioMAE-pytorch/audio_mae.py
code served (permissive licence) · get_code("c691b3b94f34ae10")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, PatchEmbed: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Construct the PatchEmbed module
model = nn.Module()

# Create a sample input tensor matching the expected dimensions
# img_size=(1024, 128), so input should be (B, C, H, W) = (B, 1, 1024, 128)
x = torch.randn(2, 1, 1024, 128)

# Define ARGS dictionary with all parameters for PatchEmbed.__init__
ARGS = {
    'img_size': (1024, 128),
    'patch_size': (16, 16),
    'in_chans': 1,
    'embed_dim': 768
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type PatchEmbed — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a PatchEmbed output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
PatchEmbed_org Ran
ran on a synthesized input · constructed an object; no result was computed
facebookresearch/audiomae/models_mae.py
pointer only (licence: NOASSERTION) · get_code("aaa7fea64dd2b57c")
How it ran

The body of this sample is withheld: GitHub could not classify the licence of this paper's repository, facebookresearch/audiomae (NOASSERTION). The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Residual Ran
ran on a synthesized input · constructed an object; no result was computed
rishikksh20/AudioMAE-pytorch/audio_mae.py
code served (permissive licence) · get_code("c824fe1a650a3f66")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Residual: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# The Residual class wraps a function/module and adds a residual connection
# It calls self.fn(x, **kwargs) and adds x to the result
# So fn must be a module that takes x and returns something of the same shape as x

class SimpleModule(nn.Module):
    """A simple module that takes input and returns output of same shape"""
    def __init__(self, in_features, out_features):
        super().__init__()
        self.linear = nn.Linear(in_features, out_features)
    
    def forward(self, x, **kwargs):
        return self.linear(x)

# Create a simple module where input and output dimensions match
# so the residual connection x + fn(x) is valid
fn_module = SimpleModule(in_features=4, out_features=4)

ARGS = {
    'fn': fn_module
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Residual — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Residual output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
SwinBlock Ran
ran on a synthesized input · constructed an object; no result was computed
rishikksh20/AudioMAE-pytorch/audio_mae.py
code served (permissive licence) · get_code("22e4ee062fba7a5e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, SwinBlock: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn
import numpy as np

torch.manual_seed(0)

ARGS = {
    "dim": 4,
    "heads": 2,
    "head_dim": 2,
    "mlp_dim": 8,
    "shifted": False,
    "window_size": 2,
    "relative_pos_embedding": True,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
einops, numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type SwinBlock — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a SwinBlock output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
SwinBlock Ran
ran on a synthesized input · constructed an object; no result was computed
eml-eda/tle-supervised/models/models_audio_mae.py
pointer only (licence: NONE) · get_code("0caaefeb6252c9e5")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, eml-eda/tle-supervised when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Transformer Ran
ran on a synthesized input · constructed an object; no result was computed
rishikksh20/AudioMAE-pytorch/audio_mae.py
code served (permissive licence) · get_code("54a9926f9e0936b8")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Transformer: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Define all parameters for Transformer.__init__
ARGS = {
    "dim": 4,
    "depth": 2,
    "heads": 2,
    "dim_head": 2,
    "mlp_dim": 8,
    "dropout": 0.1,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
einops, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Transformer — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Transformer output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Transformer Ran
ran on a synthesized input · constructed an object; no result was computed
eml-eda/tle-supervised/models/models_audio_mae.py
pointer only (licence: NONE) · get_code("8ae2d3c8074c9653")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, eml-eda/tle-supervised when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

WindowAttention Ran
ran on a synthesized input · constructed an object; no result was computed
rishikksh20/AudioMAE-pytorch/audio_mae.py
code served (permissive licence) · get_code("d7227307e8862fc8")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, WindowAttention: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn

torch.manual_seed(0)

ARGS = {
    'dim': 4,
    'heads': 2,
    'head_dim': 2,
    'shifted': True,
    'window_size': 4,
    'relative_pos_embedding': True,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
einops, numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type WindowAttention — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a WindowAttention output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
WindowAttention Ran
ran on a synthesized input · constructed an object; no result was computed
eml-eda/tle-supervised/models/models_audio_mae.py
pointer only (licence: NONE) · get_code("6322ff428c4c429a")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, eml-eda/tle-supervised when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

create_mask Ran
ran, violated the contract we drafted
rishikksh20/AudioMAE-pytorch/audio_mae.py
code served (permissive licence) · get_code("5b1dded6034b41e3")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (16, 16) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from einops import rearrange

torch.manual_seed(0)

ARGS = {
    "window_size": 4,
    "displacement": 1,
    "upper_lower": True,
    "left_right": True,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
einops, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (16, 16) — shape (16, 16) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_2d_sincos_pos_embed Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
eml-eda/tle-supervised/models/models_audio_mae.py
pointer only (licence: NONE) · get_code("11b4cbd5cc85097d")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, eml-eda/tle-supervised when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

get_relative_distances Ran
ran, honoured the contract we drafted
rishikksh20/AudioMAE-pytorch/audio_mae.py
code served (permissive licence) · get_code("ea931f8eaca3c48c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 3, dtype family int, shape (9, 9, 2) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

ARGS = {
    "window_size": 3
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 3, dtype family int, shape (9, 9, 2) — shape (9, 9, 2) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
AudioMaskedAutoencoderViT Not yet run
unverified (harvested, no recorded run)
rishikksh20/AudioMAE-pytorch/audio_mae.py
code served (permissive licence) · get_code("c263573424bacb51")
How it ran

no invocation recorded.

AudioMaskedAutoencoderViT Not yet run
unverified (harvested, no recorded run)
eml-eda/tle-supervised/models/models_audio_mae.py
pointer only (licence: NONE) · get_code("b9ab992d4f387299")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, eml-eda/tle-supervised when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

MAEOutput Not yet run
unverified (harvested, no recorded run)
facebookresearch/multimodal/torchmultimodal/models/masked_auto_encoder/model.py
code served (permissive licence) · get_code("a848037ef9265b59")
How it ran

no invocation recorded.

MaskedAutoEncoder Not yet run
unverified (harvested, no recorded run)
facebookresearch/multimodal/torchmultimodal/models/masked_auto_encoder/model.py
code served (permissive licence) · get_code("680afbcfbca11fb9")
How it ran

no invocation recorded.

MaskedAutoencoderViT Not yet run
unverified (harvested, no recorded run)
facebookresearch/audiomae/models_mae.py
pointer only (licence: NOASSERTION) · get_code("28128850f3b7974d")
How it ran

The body of this sample is withheld: GitHub could not classify the licence of this paper's repository, facebookresearch/audiomae (NOASSERTION). The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

PatchEmbed_new Not yet run
unverified (harvested, no recorded run)
facebookresearch/audiomae/models_mae.py
pointer only (licence: NOASSERTION) · get_code("d28d7b155c97688a")
How it ran

The body of this sample is withheld: GitHub could not classify the licence of this paper's repository, facebookresearch/audiomae (NOASSERTION). The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

SwinTransformer Not yet run
unverified (harvested, no recorded run)
facebookresearch/multimodal/torchmultimodal/models/masked_auto_encoder/model.py
code served (permissive licence) · get_code("fef623fdfcfbe69b")
How it ran

no invocation recorded.

SwinTransformerBlock Not yet run
unverified (harvested, no recorded run)
facebookresearch/multimodal/torchmultimodal/models/masked_auto_encoder/model.py
code served (permissive licence) · get_code("599cc34da67f0356")
How it ran

no invocation recorded.

TransformerEncoder Not yet run
unverified (harvested, no recorded run)
facebookresearch/multimodal/torchmultimodal/models/masked_auto_encoder/model.py
code served (permissive licence) · get_code("fd67d2fafb66524c")
How it ran

no invocation recorded.

TransformerEncoderLayer Not yet run
unverified (harvested, no recorded run)
facebookresearch/multimodal/torchmultimodal/models/masked_auto_encoder/model.py
code served (permissive licence) · get_code("7a00feceab29852a")
How it ran

no invocation recorded.

TransformerOutput Not yet run
unverified (harvested, no recorded run)
facebookresearch/multimodal/torchmultimodal/models/masked_auto_encoder/model.py
code served (permissive licence) · get_code("f7d645cfcd41319b")
How it ran

no invocation recorded.

WindowMultiHeadAttention Not yet run
unverified (harvested, no recorded run)
facebookresearch/multimodal/torchmultimodal/models/masked_auto_encoder/model.py
code served (permissive licence) · get_code("bcffcc5e0273f4c1")
How it ran

no invocation recorded.

audio_mae Not yet run
unverified (harvested, no recorded run)
facebookresearch/multimodal/torchmultimodal/models/masked_auto_encoder/model.py
code served (permissive licence) · get_code("8efb3224591920ef")
How it ran

no invocation recorded.

get_2d_sincos_pos_embed Not yet run
unverified (harvested, no recorded run)
rishikksh20/AudioMAE-pytorch/audio_mae.py
code served (permissive licence) · get_code("4dafab91858a28d2")
How it ran

no invocation recorded.

get_2d_sincos_pos_embed_flexible Not yet run
unverified (harvested, no recorded run)
facebookresearch/audiomae/models_mae.py
pointer only (licence: NOASSERTION) · get_code("728825dfce4fad97")
How it ran

The body of this sample is withheld: GitHub could not classify the licence of this paper's repository, facebookresearch/audiomae (NOASSERTION). The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

Also present in other papers' code

Of 32 samples lifted from this paper's linked repositories, 32 carry a function identity and 0 do not. Of the 29 distinct functions with an identity, 13 are also present in other papers whose code we hold (12 with at least one paper dated earlier, 0 the same month, 7 later) and 16 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
Residual
brick of this paper's assembly: helper
13 other papers
8 dated earlier, 5 later · 11 only as an assembly brick
1706.03762 (2017-06)
Attention Is All You Need
2010.11929 (2020-10)
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
2101.11986 (2021-01)
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet
and 10 more not named here
Transformer
brick of this paper's assembly: model
11 other papers
7 dated earlier, 4 later · 2 only as an assembly brick
2010.11929 (2020-10)
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
2101.11986 (2021-01)
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet
2103.14899 (2021-03)
CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification
and 8 more not named here
Attention
brick of this paper's assembly: model
6 other papers
3 dated earlier, 3 later · 2 only as an assembly brick
2103.11816 (2021-03)
Incorporating Convolution Designs into Visual Transformers
2103.14899 (2021-03)
CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification
2103.15691 (2021-03)
ViViT: A Video Vision Transformer
and 3 more not named here
get_relative_distances
brick of this paper's assembly: helper
6 other papers
2 dated earlier, 4 later · 2 only as an assembly brick
2103.14030 (2021-03)
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
2110.09057 (2021-10)
Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization
2303.07601 (2023-03)
V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception
and 3 more not named here
get_2d_sincos_pos_embed_flexible
brick of this paper's assembly: helper
5 other papers
5 later
2401.03497 (2024-01)
EAT: Self-Supervised Pre-Training with Efficient Audio Transformer
2403.07815 (2024-03)
Chronos: Learning the Language of Time Series
2405.00233 (2024-05)
SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound
and 2 more not named here
create_mask
brick of this paper's assembly: helper
4 other papers
2 dated earlier, 2 later
2103.14030 (2021-03)
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
2110.09057 (2021-10)
Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization
2311.17243 (2023-11)
PHG-Net: Persistent Homology Guided Medical Image Classification
and 1 more not named here
TransformerOutput
brick of this paper's assembly: helper
3 other papers
2 dated earlier, 1 later
2111.06377 (2021-11)
Masked Autoencoders Are Scalable Vision Learners
2205.01917 (2022-05)
CoCa: Contrastive Captioners are Image-Text Foundation Models
2301.12597 (2023-01)
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
CyclicShift
brick of this paper's assembly: helper
1 other paper
1 dated earlier
2103.14030 (2021-03)
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
MAEOutput
brick of this paper's assembly: model
1 other paper
1 dated earlier
2111.06377 (2021-11)
Masked Autoencoders Are Scalable Vision Learners
MLP
brick of this paper's assembly: helper
1 other paper
1 dated earlier
2205.01917 (2022-05)
CoCa: Contrastive Captioners are Image-Text Foundation Models
MaskedAutoEncoder
brick of this paper's assembly: model
1 other paper
1 dated earlier
2111.06377 (2021-11)
Masked Autoencoders Are Scalable Vision Learners
SwinBlock
brick of this paper's assembly: helper
1 other paper
1 dated earlier
2103.14030 (2021-03)
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
WindowAttention
brick of this paper's assembly: helper
1 other paper
1 dated earlier
2103.14030 (2021-03)
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

This paper studies a simple extension of image-based Masked Autoencoders (MAE) [1] to self-supervised representation learning from audio spectrograms. Following the Transformer encoder-decoder design in MAE, our Audio-MAE first encodes audio spectrogram patches with a high masking ratio, feeding only the non-masked tokens through encoder layers. The decoder then re-orders and decodes the encoded context padded with mask tokens, in order to reconstruct the input spectrogram. We find it beneficial to incorporate local window attention in the decoder, as audio spectrograms are highly correlated in local time and frequency bands. We then fine-tune the encoder with a lower masking ratio on target datasets. Empirically, Audio-MAE sets new state-of-the-art performance on six audio and speech classification tasks, outperforming other recent models that use external supervised pre-training. Our code and models is available at https://github.com/facebookresearch/AudioMAE. Transformers [2] and self-supervised learning [3, 4, 5,6,7,1] are dominating computer vision (CV) and natural language processing (NLP) research. The revolution firstly started in NLP with the invention of the Transformer architecture and self-attention [8]. Masked autoencoding with BERT [3] set a new state-of-the-art on various NLP tasks by self-supervised pre-training on large-scale language corpus. Similarly in the CV community, Vision Transformers (ViT) [9] have become popular for CV tasks, and, for self-supervised image representation learning, Masked Autoencoders (MAE) [1] have brought the CV community closer to the success of BERT in NLP. In addition to the existing masked autoencoders that can read (BERT) or see (MAE), in this work we study those that can listen. Transformer-based models have recently refreshed leaderboards for audio understanding tasks. For example, AST [10] and MBT [11] improved the audio classification performance on the AudioSet [12], Event Sound Classification [13], etc. The key technique behind this is initialization of audio model weights with ImageNet pre-trained supervised models (e.g., DeiT [14]) by deflating patch embeddings and interpolating positional embeddings for encoding audio spectrograms. However, exploiting ImageNet pre-trained models could be sub-optimal. Unlike initializing video models with weights from image models (e.g., the initial weights of I3D [15] or 3D-ResNets [16] are inflated from ImageNet pre-trained image models), there are clear and notable discrepancies between spectrograms representing audio content and natural images. It remains unclear why such heterogeneous image-toaudio transfer is useful beyond arguably similar low-level semantics such as shapes of spectrograms and shapes of visual objects. Further, any label bias would inevitably be transferred to audio models. Addressing these concerns, self-supervised audio representation learning has recently attracted much research attention. Based on BEiT [17] that learns to reconstruct image patches or learnt patch tokens, SS-AST [18] extends to the audio domain and exploits spectrograms (akin to 1-channel 2D images) and use both contrastive and reconstruction objective as self-supervision. Without using any labels, the key enabler to effective self-supervised representation learning is large-scale pre-training data. In this work we use AudioSet [12] for pre-training, a common dataset containing ∼2 million audio recordings. Performing large-scale training with Transformer architectures is challenging as self-attention in Transformers has quadratic complexity w.r.t. the length of input sequence. 36th Conference on Neural Information Processing Systems (NeurIPS 2022).

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2207.06405")
get_code_for_paper("2207.06405")
have("2207.06405")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2207.06405.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2207.06405.svg)](https://syntology.ai/paper/2207.06405)

Connect an agent — have() is free.