Jacek Tabor, Decebal Mocanu, Aleksandra Nowak, Bram Grooten
We lifted 20 functions out of this paper's own repositories and ran 5 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| alooow/fantastic_weights_paper | canonical | 5 of 12 |
| TimDettmers/sparse_learning | canonical | 0 of 8 |
| Function | Status | Where it lives |
|---|---|---|
| magnitude_redistribution | Ran | alooow/fantastic_weights_paper/ImageNet/funcs.py code served (permissive licence) · get_code("7fbf3f0c2be8ea47") |
| momentum_redistribution | Ran | alooow/fantastic_weights_paper/ImageNet/funcs.py code served (permissive licence) · get_code("85588244473b88f9") |
| nonzero_redistribution | Ran | alooow/fantastic_weights_paper/ImageNet/funcs.py code served (permissive licence) · get_code("e67bbb994ab5bae5") |
| snip_forward_conv2d | Ran | alooow/fantastic_weights_paper/ImageNet/snip.py code served (permissive licence) · get_code("12d59cf9ada8b41a") |
| snip_forward_linear | Ran | alooow/fantastic_weights_paper/ImageNet/snip.py code served (permissive licence) · get_code("94eeaaa88bd57941") |
| SNIP | Not yet run | alooow/fantastic_weights_paper/ImageNet/snip.py code served (permissive licence) · get_code("04768d98372daca0") |
| build_resnet | Not yet run | TimDettmers/sparse_learning/imagenet/tuned_resnet/resnet.py code served (permissive licence) · get_code("51a6a5c2a049e900") |
| build_resnet | Not yet run | alooow/fantastic_weights_paper/ImageNet/resnet.py code served (permissive licence) · get_code("f069023ee573fe51") |
| build_resnet | Not yet run | alooow/fantastic_weights_paper/models/imagenet_resnet.py code served (permissive licence) · get_code("61bbac980274b60d") |
| diverging_colors | Not yet run | TimDettmers/sparse_learning/plot_graphs.py code served (permissive licence) · get_code("20c4b599af53b4ed") |
| fast_collate | Not yet run | alooow/fantastic_weights_paper/imagenet_main.py code served (permissive licence) · get_code("717c76ec787ddfe7") |
| get_optimizer | Not yet run | alooow/fantastic_weights_paper/imagenet_main.py code served (permissive licence) · get_code("410b25b6d0e7a081") |
| get_train_loader | Not yet run | alooow/fantastic_weights_paper/imagenet_main.py code served (permissive licence) · get_code("04b0a36942f99acf") |
| initializations | Not yet run | alooow/fantastic_weights_paper/models/initializers.py code served (permissive licence) · get_code("39e82a7c002940de") |
| rainbow_colors | Not yet run | TimDettmers/sparse_learning/plot_graphs.py code served (permissive licence) · get_code("b8a31ec73b7a8b3c") |
| sequential_colors | Not yet run | TimDettmers/sparse_learning/plot_graphs.py code served (permissive licence) · get_code("4746465c14058f8a") |
| uniform_coverage | Not yet run | TimDettmers/sparse_learning/imagenet/baseline/parameterized_tensors.py code served (permissive licence) · get_code("bb11b6868648aca9") |
| variance_redistribution | Not yet run | TimDettmers/sparse_learning/mnist_cifar/extensions.py code served (permissive licence) · get_code("177842ec69eed409") |
| your_pruning | Not yet run | TimDettmers/sparse_learning/mnist_cifar/extensions.py code served (permissive licence) · get_code("77b9016770359ac8") |
| your_redistribution | Not yet run | TimDettmers/sparse_learning/mnist_cifar/extensions.py code served (permissive licence) · get_code("37199b84c6e58a0e") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Dynamic Sparse Training (DST) is a rapidly evolving area of research that seeks to optimize the sparse initialization of a neural network by adapting its topology during training. It has been shown that under specific conditions, DST is able to outperform dense models. The key components of this framework are the pruning and growing criteria, which are repeatedly applied during the training process to adjust the network's sparse connectivity. While the growing criterion's impact on DST performance is relatively well studied, the influence of the pruning criterion remains overlooked. To address this issue, we design and perform an extensive empirical analysis of various pruning criteria to better understand their impact on the dynamics of DST solutions. Surprisingly, we find that most of the studied methods yield similar results. The differences become more significant in the low-density regime, where the best performance is predominantly given by the simplest technique: magnitude-based pruning.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2306.12230")
get_code_for_paper("2306.12230")
have("2306.12230")
Connect an agent — have() is free.