Risheng Liu, Shangzhi Zeng, Jin Zhang, Wei Yao, Yaohua Liu
We lifted 8 functions out of this paper's own repositories and ran 7 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| vis-opt-group/sl-bamm | reimplementation | 6 of 7 |
| copy not recorded | — | 1 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| cat_list_to_tensor | Ran | this paper's copy was not recorded; identical code first harvested from junjieyang97/mrvrbo pointer only · get_code("3ed1406329714c3f") |
| cg | Ran | vis-opt-group/sl-bamm/hypergrad/CG_torch.py code served (permissive licence) · get_code("ccd4881b531f738e") |
| fixed_point | Ran | vis-opt-group/sl-bamm/hypergrad/hypergradients.py code served (permissive licence) · get_code("cd2cd81580a4b699") |
| get_outer_gradients | Ran | vis-opt-group/sl-bamm/hypergrad/hypergradients.py code served (permissive licence) · get_code("718723909e4de80c") |
| grad_unused_zero | Ran | vis-opt-group/sl-bamm/hypergrad/hypergradients.py code served (permissive licence) · get_code("1b9c1e465ef0029f") |
| loss_L1 | Ran | vis-opt-group/sl-bamm/merely_convex/strategy3.py code served (permissive licence) · get_code("452ac773b1804bc4") |
| loss_L2 | Ran | vis-opt-group/sl-bamm/merely_convex/strategy3.py code served (permissive licence) · get_code("1592bf271df76b62") |
| update_tensor_grads | Not yet run | vis-opt-group/sl-bamm/hypergrad/hypergradients.py code served (permissive licence) · get_code("43e10b662538de12") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Gradient methods have become mainstream techniques for Bi-Level Optimization (BLO) in learning fields. The validity of existing works heavily rely on either a restrictive Lower-Level Strong Convexity (LLSC) condition or on solving a series of approximation subproblems with high accuracy or both. In this work, by averaging the upper and lower level objectives, we propose a single loop Bi-level Averaged Method of Multipliers (sl-BAMM) for BLO that is simple yet efficient for large-scale BLO and gets rid of the limited LLSC restriction. We further provide non-asymptotic convergence analysis of sl-BAMM towards KKT stationary points, and the comparative advantage of our analysis lies in the absence of strong gradient boundedness assumption, which is always required by others. Thus our theory safely captures a wider variety of applications in deep learning, especially where the upper-level objective is quadratic w.r.t. the lower-level variable. Experimental results demonstrate the superiority of our method.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2302.03407")
get_code_for_paper("2302.03407")
have("2302.03407")
Connect an agent — have() is free.