We lifted 14 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| florentinguth/rainbow | canonical | 2 of 14 |
| Function | Status | Where it lives |
|---|---|---|
| conv1x1 | Ran | florentinguth/rainbow/models/resnet.py code served (permissive licence) · get_code("d9def42110729a85") |
| conv3x3 | Ran | florentinguth/rainbow/models/resnet.py code served (permissive licence) · get_code("fac5364e2f53c6db") |
| DCT_I | Not yet run | florentinguth/rainbow/models/DCT.py code served (permissive licence) · get_code("8dafc755add82efd") |
| DCT_II | Not yet run | florentinguth/rainbow/models/DCT.py code served (permissive licence) · get_code("240c75536744ea04") |
| DCT_III | Not yet run | florentinguth/rainbow/models/DCT.py code served (permissive licence) · get_code("1dbdb66cd1152c11") |
| conv2d | Not yet run | florentinguth/rainbow/models/LinearProj.py code served (permissive licence) · get_code("214f33cf948da458") |
| gaussian_window | Not yet run | florentinguth/rainbow/models/STFT.py code served (permissive licence) · get_code("de6dcf943e71f6fb") |
| get_datasets | Not yet run | florentinguth/rainbow/datasets.py code served (permissive licence) · get_code("e6fb903521a8cee4") |
| hanning_window | Not yet run | florentinguth/rainbow/models/STFT.py code served (permissive licence) · get_code("7331f2a3d29fc235") |
| modulus | Not yet run | florentinguth/rainbow/models/Analysis.py code served (permissive licence) · get_code("92d96ed40e7d0a99") |
| new_logfile | Not yet run | florentinguth/rainbow/main_block.py code served (permissive licence) · get_code("31e6ebc3a900f3de") |
| relu | Not yet run | florentinguth/rainbow/models/Analysis.py code served (permissive licence) · get_code("1a2d7d179b6fcef3") |
| resnet | Not yet run | florentinguth/rainbow/models/resnet.py code served (permissive licence) · get_code("ec4c52396ad68e3e") |
| softshrink | Not yet run | florentinguth/rainbow/models/Analysis.py code served (permissive licence) · get_code("f2a6aad1f81a2872") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
A central question in deep learning is to understand the functions learned by deep networks. What is their approximation class? Do the learned weights and representations depend on initialization? Previous empirical work has evidenced that kernels defined by network activations are similar across initializations. For shallow networks, this has been theoretically studied with random feature models, but an extension to deep networks has remained elusive. Here, we provide a deep extension of such random feature models, which we call the rainbow model. We prove that rainbow networks define deterministic (hierarchical) kernels in the infinite-width limit. The resulting functions thus belong to a data-dependent RKHS which does not depend on the weight randomness. We also verify numerically our modeling assumptions on deep CNNs trained on image classification tasks, and show that the trained networks approximately satisfy the rainbow hypothesis. In particular, rainbow networks sampled from the corresponding random feature model achieve similar performance as the trained networks. Our results highlight the central role played by the covariances of network weights at each layer, which are observed to be low-rank as a result of feature learning.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2305.18512")
get_code_for_paper("2305.18512")
have("2305.18512")
Connect an agent — have() is free.