We lifted 4 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| shc443/coveringnumber_gb | canonical | 3 of 4 |
| Function | Status | Where it lives |
|---|---|---|
| bessel_k | Ran | shc443/coveringnumber_gb/MaternKernelTorch.py code served (permissive licence) · get_code("66d7b3dcb7bee596") |
| myExpFunc | Ran | shc443/coveringnumber_gb/.ipynb_checkpoints/complexity-checkpoint.py code served (permissive licence) · get_code("b77a67b45a7d40c8") |
| nonlin | Ran | shc443/coveringnumber_gb/.ipynb_checkpoints/complexity-checkpoint.py code served (permissive licence) · get_code("e850935d498e2118") |
| load_compositional_dataset | Not yet run | shc443/coveringnumber_gb/cleaning/curse_dimensionality_sim/load_huggingface_data.py code served (permissive licence) · get_code("5c61a10c18b5e2bc") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We show that deep neural networks (DNNs) can efficiently learn any composition of functions with bounded $F_{1}$-norm, which allows DNNs to break the curse of dimensionality in ways that shallow networks cannot. More specifically, we derive a generalization bound that combines a covering number argument for compositionality, and the $F_{1}$-norm (or the related Barron norm) for large width adaptivity. We show that the global minimizer of the regularized loss of DNNs can fit for example the composition of two functions $f^{*}=h\circ g$ from a small number of observations, assuming $g$ is smooth/regular and reduces the dimensionality (e.g. $g$ could be the quotient map of the symmetries of $f^{*}$), so that $h$ can be learned in spite of its low regularity. The measures of regularity we consider is the Sobolev norm with different levels of differentiability, which is well adapted to the $F_{1}$ norm. We compute scaling laws empirically and observe phase transitions depending on whether $g$ or $h$ is harder to learn, as predicted by our theory.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2407.05664")
get_code_for_paper("2407.05664")
have("2407.05664")
Connect an agent — have() is free.