We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent. The picture can be radically different for narrow networks, which tend to get stuck in badly-generalizing local minima. Here we investigate the cross-over between these two regimes in the high-dimensional setting, and in particular investigate the connection between the so-called mean-field/hydrodynamic regime and the seminal approach of Saad & Solla. Focusing on the case of Gaussian data, we study the interplay between the learning rate, the time scale, and the number of hidden units in the high-dimensional dynamics of stochastic gradient descent (SGD). Our work builds on a deterministic description of SGD in high-dimensions from statistical physics, which we extend and for which we provide rigorous convergence rates.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2202.00293")
get_code_for_paper("2202.00293")
have("2202.00293")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2202.00293.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2202.00293)
Connect an agent — have() is free.