We lifted 8 functions out of this paper's own repositories and ran 7 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| xuzhiqin1990/F-Principle | reimplementation | 4 of 5 |
| xuzhiqin1990/mscalednn | reimplementation | 3 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| func0 | Ran | xuzhiqin1990/F-Principle/main_1d_20190924.py pointer only (licence: NONE) · get_code("c567a9e22918a4b2") |
| getWeightNorm | Ran | xuzhiqin1990/F-Principle/BasicFunc.py pointer only (licence: NONE) · get_code("eaec3a6ad43de50b") |
| getWeightNormLayer | Ran | xuzhiqin1990/F-Principle/BasicFunc.py pointer only (licence: NONE) · get_code("47f4b6968afd11b4") |
| rand_bd | Ran | xuzhiqin1990/mscalednn/exp/sin_M/62178/code.py pointer only (licence: NONE) · get_code("829510e90192ce5e") |
| rand_it | Ran | xuzhiqin1990/mscalednn/exp/sin_M/62178/code.py pointer only (licence: NONE) · get_code("06d6c667e46ee123") |
| sigmoid | Ran | xuzhiqin1990/F-Principle/main_1d_20190924.py pointer only (licence: NONE) · get_code("2d3e248be1323d2c") |
| u | Ran | xuzhiqin1990/mscalednn/exp/sin_M/62178/code.py pointer only (licence: NONE) · get_code("71f9d808a0cde4de") |
| plot_w | Not yet run | xuzhiqin1990/F-Principle/BasicFunc.py pointer only (licence: NONE) · get_code("cc48b19eaf790f9e") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We study the training process of Deep Neural Networks (DNNs) from the Fourier analysis perspective. We demonstrate a very universal Frequency Principle (F-Principle) -- DNNs often fit target functions from low to high frequencies -- on high-dimensional benchmark datasets such as MNIST/CIFAR10 and deep neural networks such as VGG16. This F-Principle of DNNs is opposite to the behavior of most conventional iterative numerical schemes (e.g., Jacobi method), which exhibit faster convergence for higher frequencies for various scientific computing problems. With a simple theory, we illustrate that this F-Principle results from the regularity of the commonly used activation functions. The F-Principle implies an implicit bias that DNNs tend to fit training data by a low-frequency function. This understanding provides an explanation of good generalization of DNNs on most real datasets and bad generalization of DNNs on parity function or randomized dataset.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("1901.06523")
get_code_for_paper("1901.06523")
have("1901.06523")
Connect an agent — have() is free.