We lifted 4 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| udc-gac/venom | canonical | 3 of 4 |
| Function | Status | Where it lives |
|---|---|---|
| nmSparsifier | Ran | udc-gac/venom/benchmark/energy.py code served (permissive licence) · get_code("866bd271c87031f8") |
| round_up | Ran | udc-gac/venom/end2end/grouped_nmv_tensor.py code served (permissive licence) · get_code("235f498ae4f93c36") |
| stringify | Ran | udc-gac/venom/benchmark/native_scripting.py code served (permissive licence) · get_code("721bc89c148723fd") |
| compile | Not yet run | udc-gac/venom/benchmark/native_scripting.py code served (permissive licence) · get_code("fea9da0420300a3a") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
The increasing success and scaling of Deep Learning models demands higher computational efficiency and power. Sparsification can lead to both smaller models as well as higher compute efficiency, and accelerated hardware is becoming available. However, exploiting it efficiently requires kernel implementations, pruning algorithms, and storage formats, to utilize hardware support of specialized sparse vector units. An example of those are the NVIDIA's Sparse Tensor Cores (SPTCs), which promise a 2x speedup. However, SPTCs only support the 2:4 format, limiting achievable sparsity ratios to 50%. We present the V:N:M format, which enables the execution of arbitrary N:M ratios on SPTCs. To efficiently exploit the resulting format, we propose Spatha, a high-performance sparse-library for DL routines. We show that Spatha achieves up to 37x speedup over cuBLAS. We also demonstrate a second-order pruning technique that enables sparsification to high sparsity ratios with V:N:M and little to no loss in accuracy in modern transformers.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2310.02065")
get_code_for_paper("2310.02065")
have("2310.02065")
Connect an agent — have() is free.