Yutaka Matsuo, Shohei Taniguchi, Yusuke Iwasawa, Masahiro Suzuki, Yuta Oshima, Keno Harada, Gouki Minegishi, Seong Jeong, Go Nagahara, Tomoshi Iiyama
We lifted 23 functions out of this paper's own repositories and ran 11 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| UiPath/torchvision | canonical | 9 of 18 |
| iShohei220/adopt | canonical | 2 of 5 |
| Function | Status | Where it lives |
|---|---|---|
| accuracy | Ran | iShohei220/adopt/imagenet/utils.py code served (permissive licence) · get_code("0d95e28fac98427f") |
| channel_shuffle | Ran | UiPath/torchvision/torchvision/models/shufflenetv2.py code served (permissive licence) · get_code("cf7081fc34608ea3") |
| conv1x1 | Ran | UiPath/torchvision/torchvision/models/resnet.py code served (permissive licence) · get_code("2a80220dabcb742a") |
| conv3x3 | Ran | UiPath/torchvision/torchvision/models/resnet.py code served (permissive licence) · get_code("600ff2c45e0de056") |
| densenet121 | Ran | UiPath/torchvision/torchvision/models/densenet.py code served (permissive licence) · get_code("9eb4b3b03ed7a1c9") |
| densenet169 | Ran | UiPath/torchvision/torchvision/models/densenet.py code served (permissive licence) · get_code("858b62a0718acf3c") |
| googlenet | Ran | UiPath/torchvision/torchvision/models/googlenet.py code served (permissive licence) · get_code("cc86d582f7c75a6d") |
| mnasnet0_75 | Ran | UiPath/torchvision/torchvision/models/mnasnet.py code served (permissive licence) · get_code("561d4224329b8802") |
| mnasnet1_0 | Ran | UiPath/torchvision/torchvision/models/mnasnet.py code served (permissive licence) · get_code("541eb8cf9b978129") |
| mobilenet_v2 | Ran | UiPath/torchvision/torchvision/models/mobilenetv2.py code served (permissive licence) · get_code("57f30bbef481b9c8") |
| store_model_weights | Ran | iShohei220/adopt/imagenet/utils.py code served (permissive licence) · get_code("d92101f375faa8a6") |
| alexnet | Not yet run | UiPath/torchvision/torchvision/models/alexnet.py code served (permissive licence) · get_code("8ccdf1f64321d691") |
| average_checkpoints | Not yet run | iShohei220/adopt/imagenet/utils.py code served (permissive licence) · get_code("4eca7815147e1708") |
| densenet161 | Not yet run | UiPath/torchvision/torchvision/models/densenet.py code served (permissive licence) · get_code("e945c7af558f65a9") |
| get_args_parser | Not yet run | iShohei220/adopt/imagenet/train_quantization.py code served (permissive licence) · get_code("e36bcfb158ae8f9c") |
| get_module | Not yet run | iShohei220/adopt/imagenet/presets.py code served (permissive licence) · get_code("b2ea55703711dd12") |
| inception_v3 | Not yet run | UiPath/torchvision/torchvision/models/inception.py code served (permissive licence) · get_code("d5f1aa4870d7ad8b") |
| load_meta_file | Not yet run | UiPath/torchvision/torchvision/datasets/imagenet.py code served (permissive licence) · get_code("fd93b84b5a89ac58") |
| mnasnet0_5 | Not yet run | UiPath/torchvision/torchvision/models/mnasnet.py code served (permissive licence) · get_code("095cb1dfd64f93c1") |
| parse_devkit_archive | Not yet run | UiPath/torchvision/torchvision/datasets/imagenet.py code served (permissive licence) · get_code("d523a6fcfaf250c7") |
| resnet18 | Not yet run | UiPath/torchvision/torchvision/models/resnet.py code served (permissive licence) · get_code("6af00ec71cf439d8") |
| shufflenet_v2_x0_5 | Not yet run | UiPath/torchvision/torchvision/models/shufflenetv2.py code served (permissive licence) · get_code("3edb47a32cd6a773") |
| shufflenet_v2_x1_0 | Not yet run | UiPath/torchvision/torchvision/models/shufflenetv2.py code served (permissive licence) · get_code("97931550315207a2") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Adam is one of the most popular optimization algorithms in deep learning. However, it is known that Adam does not converge in theory unless choosing a hyperparameter, i.e., β 2 , in a problem-dependent manner. There have been many attempts to fix the non-convergence (e.g., AMSGrad), but they require an impractical assumption that the gradient noise is uniformly bounded. In this paper, we propose a new adaptive gradient method named ADOPT, which achieves the optimal convergence rate of O(1/ √ T ) with any choice of β 2 without depending on the bounded noise assumption. ADOPT addresses the non-convergence issue of Adam by removing the current gradient from the second moment estimate and changing the order of the momentum update and the normalization by the second moment estimate. We also conduct intensive numerical experiments, and verify that our ADOPT achieves superior results compared to Adam and its variants across a wide range of tasks, including image classification, generative modeling, natural language processing, and deep reinforcement learning. The implementation is available at https://github.com/iShohei220/adopt.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2411.02853")
get_code_for_paper("2411.02853")
have("2411.02853")
Connect an agent — have() is free.