SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2304.04858 · CVPR · 2023

Simulated Annealing in Early Layers Leads to Better Generalization

Mirco Ravanelli, Eugene Belilovsky, Sudhir Mudur, Muawiz Chaudhary, Amir Sarfi, Zahra Karimpour, Nasir Khalid

arXiv · PDF · Open in the Atlas

Code that ran

21 of 37 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (5 of them constructed an object rather than computing a result; 2 honoured the contract we drafted, 0 violated it and 6 have no contract verdict; 13 failures of our instrument; 16 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2304.04858.json · badge: /api/ran/2304.04858.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
amiiir-sarfi/seal marked official in Papers with Code
Papers with Code archive
20 ran · 15 no recorded run
copy not recorded community
not in the paper's declared repositories
1 ran · 1 no recorded run
FunctionStatusWhere it lives
AverageMeter Ran
ran on a synthesized input · constructed an object; no result was computed
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("402a4933d75f91b2")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

BasicBlock Ran
ran on a synthesized input · constructed an object; no result was computed
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("77b9d81bc5e4dd19")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Bottleneck Ran
ran on a synthesized input · constructed an object; no result was computed
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("966080dd9d9fa737")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

KDLoss Ran
ran on a synthesized input
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("ae4f25439b3acbb3")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

LabelSmoothing Ran
ran on a synthesized input · constructed an object; no result was computed
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("afc0fd280b0b83c5")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

PairBatchSampler Ran
ran on a synthesized input · constructed an object; no result was computed
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("ae4960bdf070be2e")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

_get_mean_std Ran
ran, honoured the contract we drafted
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("adb312560b8eca5d")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

_warmup_lr Ran
ran, honoured the contract we drafted
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("60087ec002f63a64")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

cifar_lr Ran
ran; our contract draft was wrong, not the code
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("2d6895c1a9b20b6a")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

constant_lr Ran
ran; our contract draft was wrong, not the code
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("c0f4ef67992f303a")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

conv3x3 Ran
ran; our contract draft was wrong, not the code
this paper's copy was not recorded; identical code first harvested from ChrisMats/Res2Net
pointer only; the stored copy get_code serves, from ChrisMats/Res2Net, is pointer only (licence: NONE) · get_code("e18ed73aa89f051a")
How it ran

The body of this sample is withheld: this paper's copy of the code was not recorded (its link names no repository or source path), so no licence was read for it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

cosine_lr Ran
ran; our contract draft was wrong, not the code
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("2e32661e8d23fbbb")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

get_ascenders_descenders Ran
ran; our contract draft was wrong, not the code
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("4f6369d4ba085a0b")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

get_optimizer Ran
ran; our contract draft was wrong, not the code
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("4e2cad7e754e6954")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

get_policy Ran
ran; our contract draft was wrong, not the code
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("84121d5fa2d05bf2")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

get_transforms Ran
ran; our contract draft was wrong, not the code
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("ae403e09fc2a944f")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

long_cosine_lr Ran
ran; our contract draft was wrong, not the code
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("1a15317faad06688")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

multistep_lr Ran
ran; our contract draft was wrong, not the code
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("701988efefe74cd8")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

reinitalize_forgetting Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("7a31c2db2967e1ca")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

step_lr Ran
ran; our contract draft was wrong, not the code
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("ba2b2c184b211eff")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

val_dependent_lr Ran
ran; our contract draft was wrong, not the code
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("beefa7eb58aba900")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

IterativeTrainer Not yet run
unverified (harvested, no recorded run)
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("d7dfb645689ad0c6")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

Meter Not yet run
unverified (harvested, no recorded run)
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("de2f613a8c141264")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

ResNet Not yet run
unverified (harvested, no recorded run)
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("4da1826351727b34")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

Stage Not yet run
unverified (harvested, no recorded run)
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("0a92f9d39e77ee23")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

_reinit Not yet run
unverified (harvested, no recorded run)
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("fed6cc06ad4c821e")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

assign_learning_rate Not yet run
unverified (harvested, no recorded run)
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("047bd41c9984a7fb")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

get_model Not yet run
unverified (harvested, no recorded run)
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("492c4d6d39fb421a")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

load_ckpt Not yet run
unverified (harvested, no recorded run)
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("ac6f2ab677109bb5")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

load_dataset Not yet run
unverified (harvested, no recorded run)
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("80e07bb784b914ea")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

resnet18 Not yet run
unverified (harvested, no recorded run)
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("6433f0d1078d502e")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

resnet50 Not yet run
unverified (harvested, no recorded run)
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("0cb0dca347d472c9")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

rgetattr Not yet run
unverified (harvested, no recorded run)
this paper's copy was not recorded; identical code first harvested from opengvlab/internimage
pointer only · get_code("44925cf3831bfe79")
How it ran

The body of this sample is withheld: this paper's copy of the code was not recorded (its link names no repository or source path), so no licence was read for it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

save_checkpoint Not yet run
unverified (harvested, no recorded run)
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("6fcb2d5ec5bc611b")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

summary_logging Not yet run
unverified (harvested, no recorded run)
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("9b97b707b5baccde")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

touch_dir Not yet run
unverified (harvested, no recorded run)
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("c5d4b37a50c67113")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

wandb_logging Not yet run
unverified (harvested, no recorded run)
amiiir-sarfi/seal/core.py
pointer only (licence: NONE) · get_code("e62d3be00dcdc132")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, amiiir-sarfi/seal when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

Also present in other papers' code

Of 37 samples lifted from this paper's linked repositories, 37 carry a function identity and 0 do not. Of the 37 distinct functions with an identity, 9 are also present in other papers whose code we hold (9 with at least one paper dated earlier, 1 the same month, 4 later) and 28 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
resnet18
brick of this paper's assembly: helper
89 other papers
62 dated earlier, 1 the same month, 25 later, 1 same year, order not determinable
1705.09759 (2017-05)
CASENet: Deep Category-Aware Semantic Edge Detection
1801.07698 (2018-01)
ArcFace: Additive Angular Margin Loss for Deep Face Recognition
1803.00839 (2018-03)
Pose-Robust Face Recognition via Deep Residual Equivariant Mapping
and 86 more not named here
conv3x3
brick of this paper's assembly: model
13 other papers
11 dated earlier, 1 later, 1 same year, order not determinable · 1 only as an assembly brick
1512.03385 (2015-12)
Deep Residual Learning for Image Recognition
1904.01169 (2019-04)
Res2Net: A New Multi-scale Backbone Architecture
1907.11561 (2019-07)
Deep Learning for Classification and Severity Estimation of Coffee Leaf Biotic Stress
and 10 more not named here
resnet50
brick of this paper's assembly: helper
13 other papers
6 dated earlier, 7 later
1603.04992 (2016-03)
Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
1905.03244 (2019-05)
Convolutional Mesh Regression for Single-Image Human Shape Reconstruction
2006.05734 (2020-06)
3D Human Mesh Regression with Dense Correspondence
and 10 more not named here
rgetattr
brick of this paper's assembly: helper
13 other papers
4 dated earlier, 8 later, 1 same year, order not determinable · 1 only as an assembly brick
2203.04036 (2022-03)
StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN
2209.02952 (2022-09)
BiFuse++: Self-supervised and Efficient Bi-projection Fusion for 360 Depth Estimation
2211.05778 (2022-11)
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions
and 10 more not named here
AverageMeter
brick of this paper's assembly: helper
1 other paper
1 dated earlier · 1 only as an assembly brick
2103.05152 (2021-03)
Knowledge Evolution in Neural Networks
LabelSmoothing
brick of this paper's assembly: loss
1 other paper
1 dated earlier
2010.11929 (2020-10)
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Meter
brick of this paper's assembly: helper
1 other paper
1 dated earlier · 1 only as an assembly brick
2103.05152 (2021-03)
Knowledge Evolution in Neural Networks
_warmup_lr
brick of this paper's assembly: helper
1 other paper
1 dated earlier · 1 only as an assembly brick
2103.05152 (2021-03)
Knowledge Evolution in Neural Networks
assign_learning_rate
brick of this paper's assembly: helper
1 other paper
1 dated earlier · 1 only as an assembly brick
2103.05152 (2021-03)
Knowledge Evolution in Neural Networks

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Recently, a number of iterative learning methods have been introduced to improve generalization. These typically rely on training for longer periods of time in exchange for improved generalization. LLF (later-layer-forgetting) is a state-of-the-art method in this category. It strengthens learning in early layers by periodically re-initializing the last few layers of the network. Our principal innovation in this work is to use Simulated annealing in EArly Layers (SEAL) of the network in place of re-initialization of later layers. Essentially, later layers go through the normal gradient descent process, while the early layers go through short stints of gradient ascent followed by gradient descent. Extensive experiments on the popular Tiny-ImageNet dataset benchmark and a series of transfer learning and few-shot learning tasks show that we outperform LLF by a significant margin. We further show that, compared to normal training, LLF features, although improving on the target task, degrade the transfer learning performance across all datasets we explored. In comparison, our method outperforms LLF across the same target datasets by a large margin. We also show that the prediction depth of our method is significantly lower than that of LLF and normal training, indicating on average better prediction performance. 1

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2304.04858")
get_code_for_paper("2304.04858")
have("2304.04858")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2304.04858.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2304.04858.svg)](https://syntology.ai/paper/2304.04858)

Connect an agent — have() is free.