SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2111.00899 · ICLR · 2021

Equivariant Contrastive Learning

Brian Cheung, Rumen Dangovski, Jing Li, Charlotte Loh, Seungwook Han, Akash Srivastava, Marin Soljačić

arXiv · PDF · Open in the Atlas

Code that ran

8 of 8 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (8 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 0 violated it and 8 have no contract verdict; 0 failures of our instrument; 0 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2111.00899.json · badge: /api/ran/2111.00899.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
rdangovs/essl canonical (classifier label)
classifier basis: classifier_2025
8 ran · 0 no recorded run
FunctionStatusWhere it lives
Clas Ran
ran on a synthesized input · constructed an object; no result was computed
rdangovs/essl/photonics/ssl_mode.py
code served (permissive licence) · get_code("dfdd33a3aaddf359")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Clas: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Define arguments for Clas.__init__
Lvpj = [8]  # List of layer dimensions for hidden layers
latent_dim = 4  # Input dimension (small for testing)
linear = False  # Use non-linear architecture
bnorm = True  # Use batch normalization
nrot = 4  # Number of output classes/rotations
depth = 2  # Network depth

ARGS = {
    "Lvpj": Lvpj,
    "latent_dim": latent_dim,
    "linear": linear,
    "bnorm": bnorm,
    "nrot": nrot,
    "depth": depth,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Clas — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Clas output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Classifier Ran
ran on a synthesized input · constructed an object; no result was computed
rdangovs/essl/photonics/ssl_mode.py
code served (permissive licence) · get_code("32d8df337a54368a")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Classifier: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    "Lvpj": [64],
    "in_dim": 8,
    "linear": False,
    "bnorm": True,
    "nrot": 4,
    "depth": 2,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Classifier — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Classifier output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Enc Ran
ran on a synthesized input · constructed an object; no result was computed
rdangovs/essl/photonics/ssl_mode.py
code served (permissive licence) · get_code("55808eff2e0f005f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Enc: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import math
from torch import nn

torch.manual_seed(0)

# Define ARGS dict with parameters for Enc.__init__
ARGS = {
    'Lv': [16, 32, 64, 128, 256],  # Channel dimensions for layers
    'ks': 3  # Kernel size
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Enc — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Enc output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Encoder Ran
ran on a synthesized input · constructed an object; no result was computed
rdangovs/essl/photonics/ssl_mode.py
code served (permissive licence) · get_code("9769fba90ffd6f95")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Encoder: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import math
from torch import nn

torch.manual_seed(0)

# Encoder expects:
# - Lv: a list of 5 integers representing channel dimensions
# - ks: kernel size (integer)

ARGS = {
    'Lv': [16, 32, 64, 128, 256],
    'ks': 3
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Encoder — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Encoder output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
NTXentLoss Ran
ran on a synthesized input · constructed an object; no result was computed
rdangovs/essl/photonics/ssl_mode.py
code served (permissive licence) · get_code("fac76267546bbb2e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, NTXentLoss: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn
import numpy as np

torch.manual_seed(0)

class NTXentLoss(nn.Module):
    """ this implements the Normalized Temperature Cross Entropy Loss use to
    compute Contrastive Loss in the SimCLR paper.
    This code is adapted from https://github.com/sthalles/SimCLR"""

    def __init__(self, device, batch_size, temperature):
        super(NTXentLoss, self).__init__()
        self.batch_size = batch_size
        self.temperature = temperature
        self.device = device
        self.softmax = nn.Softmax(dim=-1)
        self.mask_same = self._get_correlated_mask().type(torch.bool)
        self.similarity_function = nn.CosineSimilarity(dim=-1)
        self.criterion = nn.CrossEntropyLoss(reduction='sum')

    def _get_correlated_mask(self):
        diag = np.eye(2 * self.batch_size)
        l1 = np.eye(2 * self.batch_size, 2 * self.batch_size, k=-self.batch_size)
        l2 = np.eye(2 * self.batch_size, 2 * self.batch_size, k=self.batch_size)
        mask = torch.from_numpy(diag + l1 + l2)
        mask = (1 - mask).type(torch.bool)
        return mask.to(self.device)

    def forward(self, zis, zjs=None):
        representations = torch.cat([zjs, zis], dim=0)
        similarity_matrix = self.similarity_function(representations.unsqueeze(0), representations.unsqueeze(1))
        l_pos = torch.diag(similarity_matrix, self.batch_size)
        r_pos = torch.diag(similarity_matrix, -self.batch_size)
        positives = torch.cat([l_pos, r_pos]).view(2 * self.batch_size, 1)
        negatives = similarity_matrix[self.mask_same].view(2 * self.batch_size, -1)
        logits = torch.cat((positives, negatives), dim=1)
        logits /= self.temperature
        labels = torch.zeros(2 * self.batch_size).to(self.device).long()
        loss = self.criterion(logits, labels)
        return loss / (2 * self.batch_size)

device = 'cpu'
batch_size = 2
temperature = 0.07

ARGS = {
    'device': device,
    'batch_size': batch_size,
    'temperature': temperature
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type NTXentLoss — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a NTXentLoss output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Proj Ran
ran on a synthesized input · constructed an object; no result was computed
rdangovs/essl/photonics/ssl_mode.py
code served (permissive licence) · get_code("c91971a4e4ce87ac")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Proj: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Define arguments for Proj.__init__
Lvpj = [8, 4]  # Two distinct dimensions: hidden layer size and output size
latent_dim = 6  # Distinct from Lvpj dimensions
bnorm = False
depth = 2

ARGS = {
    "Lvpj": Lvpj,
    "latent_dim": latent_dim,
    "bnorm": bnorm,
    "depth": depth,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Proj — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Proj output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Projector Ran
ran on a synthesized input · constructed an object; no result was computed
rdangovs/essl/photonics/ssl_mode.py
code served (permissive licence) · get_code("cdce77ee6af21aa6")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Projector: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn

torch.manual_seed(0)

# Lvpj is a list with two elements: [hidden_dim_intermediate, output_dim]
# The constructor uses Lvpj[0] and Lvpj[1] to define layer dimensions
Lvpj = [8, 6]

# hidden_dim is the input dimension to the first Linear layer
hidden_dim = 4

# bnorm controls whether BatchNorm1d is applied
bnorm = False

# depth controls how many intermediate layers are created
depth = 2

ARGS = {
    "Lvpj": Lvpj,
    "hidden_dim": hidden_dim,
    "bnorm": bnorm,
    "depth": depth,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Projector — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Projector output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
SimCLRwEE Ran
ran on a synthesized input · constructed an object; no result was computed
rdangovs/essl/photonics/ssl_mode.py
code served (permissive licence) · get_code("3fb05602a98575a6")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, SimCLRwEE: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
import math

torch.manual_seed(0)

# Define small stand-in classes and build minimal valid arguments

# Lv: list of layer dimensions for Encoder
Lv = [8, 16, 32, 64, 128]

# Lvpj: list of layer dimensions for Projector
Lvpj = [64, 32]

# ks: kernel size for Conv2d
ks = 3

# device
device = torch.device('cpu')

# batchsize_ptxt: batch size for pretext task
batchsize_ptxt = 2

# temperature: temperature parameter for NTXentLoss
temperature = 0.07

# use_projector: boolean flag
use_projector = True

# nrot: number of rotations for classifier
nrot = 4

# depth: depth parameter for projector
depth = 3

# scdepth: depth for self-classifier
scdepth = 3

# eeloss: loss function type
eeloss = 'xent'

# relative_ee: boolean flag
relative_ee = False

ARGS = {
    'Lv': Lv,
    'Lvpj': Lvpj,
    'ks': ks,
    'device': device,
    'batchsize_ptxt': batchsize_ptxt,
    'temperature': temperature,
    'use_projector': use_projector,
    'nrot': nrot,
    'depth': depth,
    'scdepth': scdepth,
    'eeloss': eeloss,
    'relative_ee': relative_ee
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type SimCLRwEE — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a SimCLRwEE output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block

Also present in other papers' code

Of 8 samples lifted from this paper's linked repositories, 8 carry a function identity and 0 do not. Of the 8 distinct functions with an identity, 0 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 0 later) and 8 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

In state-of-the-art self-supervised learning (SSL) pre-training produces semantically good representations by encouraging them to be invariant under meaningful transformations prescribed from human knowledge. In fact, the property of invariance is a trivial instance of a broader class called equivariance, which can be intuitively understood as the property that representations transform according to the way the inputs transform. Here, we show that rather than using only invariance, pre-training that encourages non-trivial equivariance to some transformations, while maintaining invariance to other transformations, can be used to improve the semantic quality of representations. Specifically, we extend popular SSL methods to a more general framework which we name Equivariant Self-Supervised Learning (E-SSL). In E-SSL, a simple additional pre-training objective encourages equivariance by predicting the transformations applied to the input. We demonstrate E-SSL's effectiveness empirically on several popular computer vision benchmarks, e.g. improving SimCLR to 72.5% linear probe accuracy on ImageNet. Furthermore, we demonstrate usefulness of E-SSL for applications beyond computer vision; in particular, we show its utility on regression problems in photonics science. Our code, datasets and pre-trained models are available at https://github.com/rdangovs/essl to aid further research in E-SSL.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2111.00899")
get_code_for_paper("2111.00899")
have("2111.00899")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2111.00899.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2111.00899.svg)](https://syntology.ai/paper/2111.00899)

Connect an agent — have() is free.