SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2010.07962 · ICML · 2021

Bilevel Optimization: Convergence Analysis and Enhanced Design

Yingbin Liang, Kaiyi Ji, Junjie Yang

arXiv · PDF · Open in the Atlas

Code that ran

14 of 18 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 2 honoured the contract we drafted, 0 violated it and 4 have no contract verdict; 8 failures of our instrument; 4 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2010.07962.json · badge: /api/ran/2010.07962.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
junjieyang97/stocbio_hp canonical (classifier label)
classifier basis: census
11 ran · 3 no recorded run
JunjieYang97/stocBiO canonical (classifier label)
classifier basis: classifier_2025
3 ran · 1 no recorded run
FunctionStatusWhere it lives
accuracy Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
junjieyang97/stocbio_hp/Meta-learning/ANIL/miniimagenet/anil.py
code served (permissive licence) · get_code("d41eb4e2f3bebbf1")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# The function expects:
# - predictions: a tensor with shape (batch_size, num_classes) that will be argmaxed along dim=1
# - targets: a tensor with shape (batch_size,) containing class indices

batch_size = 2
num_classes = 4

# predictions: logits tensor of shape (batch_size, num_classes)
predictions = torch.randn(batch_size, num_classes)

# targets: ground truth class indices of shape (batch_size,)
targets = torch.tensor([0, 2])

ARGS = {
    "predictions": predictions,
    "targets": targets,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
cat_list_to_tensor Ran
ran, honoured the contract we drafted
junjieyang97/stocbio_hp/Hyperparameter-optimization/hypergrad/CG_torch.py
code served (permissive licence) · get_code("3ed1406329714c3f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned Tensor, ndim 1, dtype family float, shape (29,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# The function expects list_tx to be a list of tensors
# It flattens each tensor and concatenates them
list_tx = [
    torch.randn(2, 4),      # First tensor: shape (2, 4)
    torch.randn(3, 5),      # Second tensor: shape (3, 5)
    torch.randn(6),         # Third tensor: shape (6,)
]

ARGS = {
    "list_tx": list_tx,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 1, dtype family float, shape (29,) — shape (29,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
cg Ran
ran; our contract draft was wrong, not the code
junjieyang97/stocbio_hp/Hyperparameter-optimization/hypergrad/CG_torch.py
code served (permissive licence) · get_code("31f282d2a7c181bb")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned list. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Define a simple linear operator function that takes a list of tensors
# and returns a list of tensors (simulating matrix-vector multiplication)
def linear_operator(x_list):
    """
    Simple linear operator: Ax where A is a small matrix.
    Takes list of tensors, returns list of tensors.
    """
    result = []
    for x in x_list:
        # Simple operation: multiply by 2 (acts like a diagonal matrix)
        result.append(2.0 * x)
    return result

# Create the b vector as a list of small tensors
b = [
    torch.randn(4),
    torch.randn(3),
]

ARGS = {
    "Ax": linear_operator,
    "b": b,
    "max_iter": 100,
    "epsilon": 1e-05,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list — a list output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
fixed_point Ran
ran; our contract draft was wrong, not the code
junjieyang97/stocbio_hp/Hyperparameter-optimization/hypergrad/hypergradients.py
code served (permissive licence) · get_code("11d924c279d6166a")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from typing import List, Callable

torch.manual_seed(0)

# Define a simple inner problem: minimize ||w - target||^2
def create_fp_map(target: torch.Tensor) -> Callable:
    """Creates a fixed point map for a simple quadratic problem."""
    def fp_map(params: List[torch.Tensor], hparams: List[torch.Tensor]) -> List[torch.Tensor]:
        # Simple fixed point: w_new = w - lr * (w - target)
        # where lr is controlled by hparams
        lr = hparams[0]
        w_new = params[0] - lr * (params[0] - target)
        return [w_new]
    return fp_map

def create_outer_loss(target: torch.Tensor) -> Callable:
    """Creates an outer loss function."""
    def outer_loss(params: List[torch.Tensor], hparams: List[torch.Tensor]) -> torch.Tensor:
        # Outer loss: ||params - target||^2
        return torch.sum((params[0] - target) ** 2)
    return outer_loss

# Set up parameters (inner loop variables)
params = [torch.randn(2, 4, requires_grad=False)]

# Set up hyperparameters (outer loop variables)
hparams = [torch.tensor(0.1, requires_grad=True)]

# Set up the fixed point map and outer loss
target = torch.randn(2, 4)
fp_map = create_fp_map(target)
outer_loss = create_outer_loss(target)

# Number of fixed point iterations
K = 5

# Tolerance for early stopping
tol = 1e-10

# Whether to set gradients on hparams
set_grad = True

# Whether the fixed point map is stochastic
stochastic = False

ARGS = {
    "params": params,
    "hparams": hparams,
    "K": K,
    "fp_map": fp_map,
    "outer_loss": outer_loss,
    "tol": tol,
    "set_grad": set_grad,
    "stochastic": stochastic,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
from_sparse Ran
ran on a synthesized input
junjieyang97/stocbio_hp/Hyperparameter-optimization/experimental/l2reg_on_twentynews.py
code served (permissive licence) · get_code("e73c33a4af5fe161")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
from scipy import sparse

torch.manual_seed(0)

# Create a small sparse matrix
# The function expects x to have .tocoo() method, so we use scipy.sparse
row = np.array([0, 1, 2])
col = np.array([0, 2, 1])
data = np.array([1.0, 2.0, 3.0])
sparse_matrix = sparse.coo_matrix((data, (row, col)), shape=(3, 4))

ARGS = {
    "x": sparse_matrix
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, scipy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, shape (3, 4) — shape (3, 4) as a later run recorded it
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
gd_step Ran
ran; our contract draft was wrong, not the code
junjieyang97/stocbio_hp/Hyperparameter-optimization/hypergrad/diff_optimizers.py
pointer only (licence: MIT); the stored copy get_code serves, from cats1520cakes/Q-Detection-A-Quantum-Classical-Hybrid-Poisoning-Attack-Detection-Method, is pointer only (licence: NONE) · get_code("3ffa7e0c883965f8")
How it ran

The body of this sample is withheld: this paper's link allows the body inline (MIT), but get_code serves the stored copy, first harvested from cats1520cakes/Q-Detection-A-Quantum-Classical-Hybrid-Poisoning-Attack-Detection-Method, and GitHub detected no licence in that repository when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

gradient_fy Ran
ran; our contract draft was wrong, not the code
JunjieYang97/stocBiO/Hyperparameter-optimization/experimental/stocBiO.py
code served (permissive licence) · get_code("b233bbdedd873c21")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned Tensor, ndim 2, dtype family float, shape (3, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from torch.nn import functional as F

torch.manual_seed(0)

# Create a small neural network that produces output compatible with cross_entropy
class SimpleModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(4, 3)
    
    def forward(self, x):
        return self.linear(x)

# Create model and get a parameter to compute gradients with respect to
model = SimpleModel()
params = model.linear.weight

# Create small input data
args = torch.randn(2, 4)  # batch_size=2, input_dim=4

# Create labels for cross_entropy (class indices)
labels = torch.tensor([0, 2])

# Create output from the model
output = model(args)

# Ensure params requires grad for autograd to work
params.requires_grad_(True)

ARGS = {
    "args": args,
    "labels": labels,
    "params": params,
    "data": None,
    "output": output
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (3, 4) — shape (3, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
gradient_gy Ran
ran; our contract draft was wrong, not the code
JunjieYang97/stocBiO/Hyperparameter-optimization/experimental/stocBiO.py
code served (permissive licence) · get_code("cda14929afcae4b3")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned Tensor, ndim 2, dtype family float, shape (3, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from torch.nn import functional as F

torch.manual_seed(0)

# Create a small model for params
class SimpleModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(4, 3)
    
    def forward(self, x):
        return self.linear(x)

model = SimpleModel()

# Extract params - need to be a tensor or list of tensors that requires grad
params = model.linear.weight

# Create small tensors
batch_size = 2
input_dim = 4
num_classes = 3

# args - appears to be unused in the function body, but we need to provide it
args = None

# labels_cp - labels for cross_entropy, shape (batch_size,)
labels_cp = torch.tensor([0, 2], dtype=torch.long)

# data - appears unused in function body
data = None

# hparams - passed to reg_f, can be a dict or simple object
hparams = {"lambda": 0.01}

# output - logits from model, shape (batch_size, num_classes)
output = torch.randn(batch_size, num_classes, requires_grad=True)

# reg_f - a regularization function that takes (params, hparams, loss) and returns a scalar
def reg_f(params, hparams, loss):
    # Simple L2 regularization combined with loss
    reg_term = hparams.get("lambda", 0.01) * torch.sum(params ** 2)
    return torch.mean(loss) + reg_term

ARGS = {
    "args": args,
    "labels_cp": labels_cp,
    "params": params,
    "data": data,
    "hparams": hparams,
    "output": output,
    "reg_f": reg_f,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (3, 4) — shape (3, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
heavy_ball_step Ran
ran on a synthesized input
junjieyang97/stocbio_hp/Hyperparameter-optimization/hypergrad/diff_optimizers.py
code served (permissive licence) · get_code("d533144f406f4f80")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create small tensors for parameters
# These need to require grad so autograd.grad works
param1 = torch.tensor([1.0, 2.0, 3.0, 4.0], requires_grad=True)
param2 = torch.tensor([5.0, 6.0, 7.0, 8.0], requires_grad=True)
params = [param1, param2]

# Create auxiliary parameters (previous parameter values for momentum)
aux_param1 = torch.tensor([0.9, 1.9, 2.9, 3.9], requires_grad=False)
aux_param2 = torch.tensor([4.9, 5.9, 6.9, 7.9], requires_grad=False)
aux_params = [aux_param1, aux_param2]

# Create a simple scalar loss that depends on params
loss = (params[0] ** 2).sum() + (params[1] ** 2).sum()

# Define other arguments
step_size = 0.01
momentum = 0.9
create_graph = True

ARGS = {
    'params': params,
    'aux_params': aux_params,
    'loss': loss,
    'step_size': step_size,
    'momentum': momentum,
    'create_graph': create_graph
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
loss_train_avg Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
JunjieYang97/stocBiO/Hyperparameter-optimization/experimental/mnist_exp.py
code served (permissive licence) · get_code("d46ee4443d27d233")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned Tensor, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from torch.utils.data import DataLoader

torch.manual_seed(0)

# Create a tiny dataset: 2 samples, 28x28 images (flattened to 784)
# Labels are class indices (0-9 for MNIST-like)
images = torch.randn(2, 1, 28, 28)
labels = torch.tensor([0, 1])

# Create a simple data loader
data_loader = [(images, labels)]

# Device (CPU only)
device = torch.device('cpu')

# Parameters: shape (10, 785) where 10 is num_classes
# First 784 columns are weights, last column is bias
parameters = torch.randn(10, 785, device=device, requires_grad=True)

# Batch number to process
batch_num = 1

ARGS = {
    'data_loader': data_loader,
    'parameters': parameters,
    'device': device,
    'batch_num': batch_num
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
reverse Ran
ran, honoured the contract we drafted
junjieyang97/stocbio_hp/Hyperparameter-optimization/hypergrad/hypergradients.py
code served (permissive licence) · get_code("52b6f88728db5c40")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from typing import List, Callable

torch.manual_seed(0)

# Define a simple update map function that takes params and hparams and returns updated params
def simple_update_map(params: List[torch.Tensor], hparams: List[torch.Tensor]) -> List[torch.Tensor]:
    """Simple update map: w_new = w - lr * grad, where lr is a hyperparameter"""
    # Simulate a gradient step with learning rate from hparams
    lr = hparams[0]
    updated = []
    for p in params:
        # Create a dummy gradient
        dummy_grad = torch.ones_like(p) * 0.1
        updated.append(p - lr * dummy_grad)
    return updated

# Define outer loss function
def outer_loss_fn(params: List[torch.Tensor], hparams: List[torch.Tensor]) -> torch.Tensor:
    """Compute outer loss as sum of squared parameters"""
    loss = torch.tensor(0.0, requires_grad=True)
    for p in params:
        loss = loss + (p ** 2).sum()
    return loss

# Create parameter history: list of parameter snapshots at different iterations
# Each snapshot is a list of tensors
params_history = [
    [torch.randn(2, 4, requires_grad=True), torch.randn(3, 4, requires_grad=True)],
    [torch.randn(2, 4, requires_grad=True), torch.randn(3, 4, requires_grad=True)],
    [torch.randn(2, 4, requires_grad=True), torch.randn(3, 4, requires_grad=True)],
]

# Create hyperparameters (must have requires_grad=True)
hparams = [
    torch.tensor(0.01, requires_grad=True),
    torch.tensor(0.1, requires_grad=True),
]

# Create update map history: list of update functions
# One update map for each transition in params_history
update_map_history = [
    simple_update_map,
    simple_update_map,
    simple_update_map,
]

# Define outer loss function that will be called
outer_loss = outer_loss_fn

ARGS = {
    "params_history": params_history,
    "hparams": hparams,
    "update_map_history": update_map_history,
    "outer_loss": outer_loss,
    "set_grad": True,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
reverse_unroll Ran
ran; our contract draft was wrong, not the code
junjieyang97/stocbio_hp/Hyperparameter-optimization/hypergrad/hypergradients.py
code served (permissive licence) · get_code("677c8518ad1a22c9")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import Tensor
from typing import List, Callable

torch.manual_seed(0)

# Create hyperparameters (outer variables)
# These need requires_grad=True to compute hypergradients
hparams = [
    torch.tensor([1.0, 2.0, 3.0, 4.0], requires_grad=True),
    torch.tensor([0.5, 1.5], requires_grad=True)
]

# Create parameters (output of inner solver)
# These depend on hparams in the computation graph
params = [
    torch.tensor([2.0, 3.0, 4.0, 5.0], requires_grad=True),
    torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
]

# Ensure params depend on hparams in the graph
params[0] = params[0] + hparams[0].sum() * 0.1
params[1] = params[1] + hparams[1].sum() * 0.1

# Define outer loss function
# Takes params and hparams, returns a scalar loss
def outer_loss(params: List[Tensor], hparams: List[Tensor]) -> Tensor:
    # Simple outer objective: sum of squared parameters plus regularization on hparams
    loss = torch.sum(params[0] ** 2) + torch.sum(params[1] ** 2)
    loss = loss + 0.01 * (torch.sum(hparams[0] ** 2) + torch.sum(hparams[1] ** 2))
    return loss

ARGS = {
    "params": params,
    "hparams": hparams,
    "outer_loss": outer_loss,
    "set_grad": True
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
torch_momentum_step Ran
ran on a synthesized input
junjieyang97/stocbio_hp/Hyperparameter-optimization/hypergrad/diff_optimizers.py
code served (permissive licence) · get_code("6cdf7a8f1301bf5c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create small tensors that require gradients for computing gradients through them
params = [
    torch.randn(4, 3, requires_grad=True),
    torch.randn(2, requires_grad=True)
]

# Auxiliary parameters (velocity terms for momentum, same shape as params)
aux_params = [
    torch.zeros(4, 3),
    torch.zeros(2)
]

# Create a simple loss by combining params
loss = (params[0].sum() + params[1].sum()) ** 2

# Step size for gradient descent
step_size = 0.01

# Momentum coefficient
momentum = 0.9

# create_graph flag for higher-order derivatives
create_graph = True

ARGS = {
    "params": params,
    "aux_params": aux_params,
    "loss": loss,
    "step_size": step_size,
    "momentum": momentum,
    "create_graph": create_graph
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
train_loss Ran
ran on a synthesized input
junjieyang97/stocbio_hp/Hyperparameter-optimization/experimental/l2reg_on_twentynews.py
code served (permissive licence) · get_code("fae35b77af7219e9")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Create small tensors for the batch
batch_size = 2
input_dim = 4
num_classes = 3

x_mb = torch.randn(batch_size, input_dim)
y_mb = torch.randint(0, num_classes, (batch_size,))

# params should be a list with at least [weight, bias]
# Based on out_f: out = x @ params[0] + params[1]
# x_mb is (batch_size, input_dim), so params[0] should be (input_dim, num_classes)
weight = torch.randn(input_dim, num_classes)
bias = torch.randn(num_classes)
params = [weight, bias]

# l2_reg_params should be a 1D tensor that can be unsqueezed to shape (C, 1)
# where C is the number of classes (to match params[0] size of (input_dim, num_classes))
l2_reg_params = torch.randn(input_dim)

# l1_reg_params is optional, also 1D tensor same size as l2_reg_params
l1_reg_params = torch.randn(input_dim)

# hparams should unpack to (l2_reg_params, l1_reg_params)
hparams = (l2_reg_params, l1_reg_params)

# data should be a tuple of (x_mb, y_mb)
data = (x_mb, y_mb)

ARGS = {
    "params": params,
    "hparams": hparams,
    "data": data
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
fast_adapt Not yet run
unverified (harvested, no recorded run)
junjieyang97/stocbio_hp/Meta-learning/ANIL/miniimagenet/anil.py
code served (permissive licence) · get_code("186dcd090e91bb0e")
How it ran

no invocation recorded.

fast_adapt Not yet run
unverified (harvested, no recorded run)
junjieyang97/stocbio_hp/Meta-learning/FO-ITD-BiO/miniimagenet/ITD-BiO.py
code served (permissive licence) · get_code("e77a457bca6c4a2d")
How it ran

no invocation recorded.

get_data_loaders Not yet run
unverified (harvested, no recorded run)
junjieyang97/stocbio_hp/Hyperparameter-optimization/experimental/mnist_exp.py
code served (permissive licence) · get_code("3851e32f29e94b92")
How it ran

no invocation recorded.

stocbio Not yet run
unverified (harvested, no recorded run)
JunjieYang97/stocBiO/Hyperparameter-optimization/experimental/stocBiO.py
code served (permissive licence) · get_code("e519fae7e667e28f")
How it ran

no invocation recorded.

Also present in other papers' code

Of 18 samples lifted from this paper's linked repositories, 18 carry a function identity and 0 do not. Of the 18 distinct functions with an identity, 12 are also present in other papers whose code we hold (9 with at least one paper dated earlier, 0 the same month, 10 later) and 6 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
cat_list_to_tensor 6 other papers
3 dated earlier, 3 later · 2 only as an assembly brick
1803.06396 (2018-03)
Reviving and Improving Recurrent Back-Propagation
1909.04630 (2019-09)
Meta-Learning with Implicit Gradients
2006.14606 (2020-06)
Global Convergence and Generalization Bound of Gradient-Based Meta-Learning with Deep Neural Nets
and 3 more not named here
fixed_point 4 other papers
2 dated earlier, 2 later
1909.04630 (2019-09)
Meta-Learning with Implicit Gradients
2006.16218 (2020-06)
On the Iteration Complexity of Hypergradient Computation
2106.04692 (2021-06)
Provably Faster Algorithms for Bilevel Optimization
and 1 more not named here
reverse 3 other papers
2 dated earlier, 1 later
1909.04630 (2019-09)
Meta-Learning with Implicit Gradients
2006.16218 (2020-06)
On the Iteration Complexity of Hypergradient Computation
2106.04692 (2021-06)
Provably Faster Algorithms for Bilevel Optimization
reverse_unroll 3 other papers
2 dated earlier, 1 later
1909.04630 (2019-09)
Meta-Learning with Implicit Gradients
2006.16218 (2020-06)
On the Iteration Complexity of Hypergradient Computation
2106.04692 (2021-06)
Provably Faster Algorithms for Bilevel Optimization
accuracy 2 other papers
1 dated earlier, 1 later
2008.12284 (2020-08)
learn2learn: A Library for Meta-Learning Research
2208.10559 (2022-08)
Transductive Decoupled Variational Inference for Few-Shot Classification
cg 2 other papers
1 dated earlier, 1 later
2006.14606 (2020-06)
Global Convergence and Generalization Bound of Gradient-Based Meta-Learning with Deep Neural Nets
2106.04692 (2021-06)
Provably Faster Algorithms for Bilevel Optimization
gd_step 2 other papers
1 dated earlier, 1 later
2006.14606 (2020-06)
Global Convergence and Generalization Bound of Gradient-Based Meta-Learning with Deep Neural Nets
2507.06262 (2025-07)
Q-Detection: A Quantum-Classical Hybrid Poisoning Attack Detection Method
gradient_fy
brick of this paper's assembly: helper
1 other paper
1 later · 1 only as an assembly brick
2106.04692 (2021-06)
Provably Faster Algorithms for Bilevel Optimization
gradient_gy
brick of this paper's assembly: loss
1 other paper
1 later · 1 only as an assembly brick
2106.04692 (2021-06)
Provably Faster Algorithms for Bilevel Optimization
heavy_ball_step 1 other paper
1 dated earlier
2006.14606 (2020-06)
Global Convergence and Generalization Bound of Gradient-Based Meta-Learning with Deep Neural Nets
loss_train_avg 1 other paper
1 later
2106.04692 (2021-06)
Provably Faster Algorithms for Bilevel Optimization
torch_momentum_step 1 other paper
1 dated earlier
2006.14606 (2020-06)
Global Convergence and Generalization Bound of Gradient-Based Meta-Learning with Deep Neural Nets

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Bilevel optimization has arisen as a powerful tool for many machine learning problems such as meta-learning, hyperparameter optimization, and reinforcement learning. In this paper, we investigate the nonconvex-strongly-convex bilevel optimization problem. For deterministic bilevel optimization, we provide a comprehensive convergence rate analysis for two popular algorithms respectively based on approximate implicit differentiation (AID) and iterative differentiation (ITD). For the AID-based method, we orderwisely improve the previous convergence rate analysis due to a more practical parameter selection as well as a warm start strategy, and for the ITD-based method we establish the first theoretical convergence rate. Our analysis also provides a quantitative comparison between ITD and AID based approaches. For stochastic bilevel optimization, we propose a novel algorithm named stocBiO, which features a sample-efficient hypergradient estimator using efficient Jacobian-and Hessianvector product computations. We provide the convergence rate guarantee for stocBiO, and show that stocBiO outperforms the best known computational complexities orderwisely with respect to the condition number κ and the target accuracy . We further validate our theoretical results and demonstrate the efficiency of bilevel optimization algorithms by the experiments on meta-learning and hyperparameter optimization. Algorithm Gc(f, ) Gc(g, ) JV(g, ) HV(g, ) Gc(f, ) and Gc(g, ): number of gradient evaluations w.r.t. f and g. κ : condition number. JV(g, ): number of Jacobian-vector products ∇ x ∇ y g(x, y)v. Notation O: omit log 1 terms. HV(g, ): number of Hessian-vector products ∇ 2 y g(x, y)v. Table 2. Comparison of bilevel stochastic optimization algorithms. Algorithm Gc(F, ) Gc(G, ) JV(G, ) HV(G, ) TTSA (Hong et al., 2020) O(poly(κ) -5 2 use poly(κ) because Hong et al. 2020 does not provide the explicit dependence on κ. bilevel optimizers for the deterministic setting, and proposes a novel algorithm for the stochastic setting with order-level lower computational complexity than the existing results.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2010.07962")
get_code_for_paper("2010.07962")
have("2010.07962")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2010.07962.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2010.07962.svg)](https://syntology.ai/paper/2010.07962)

Connect an agent — have() is free.