Gunnar Rätsch, Bernhard Schölkopf, Mark Van Der Wilk, Alexander Immer, Tycho Van Der Ouderaa
We lifted 5 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| AlexImmer/ntk-marglik | canonical | 2 of 3 |
| aleximmer/ntk-marglik | canonical | 2 of 2 |
| Function | Status | Where it lives |
|---|---|---|
| expand_prior_precision | Ran | AlexImmer/ntk-marglik/ntkmarglik/marglik.py pointer only (licence: NONE) · get_code("64a6789a3a000193") |
| get_frame | Ran | aleximmer/ntk-marglik/generate_illustration_figures.py pointer only (licence: NONE) · get_code("dc30499117b3d20a") |
| get_prior_hyperparams | Ran | AlexImmer/ntk-marglik/ntkmarglik/marglik.py pointer only (licence: NONE) · get_code("e58f08a5db6faad6") |
| get_runtime_frame | Ran | aleximmer/ntk-marglik/generate_illustration_figures.py pointer only (licence: NONE) · get_code("a11cbf1c7fbeb03b") |
| valid_performance | Not yet run | AlexImmer/ntk-marglik/ntkmarglik/marglik.py pointer only (licence: NONE) · get_code("a8cb194eb229f7e7") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Selecting hyperparameters in deep learning greatly impacts its effectiveness but requires manual effort and expertise. Recent works show that Bayesian model selection with Laplace approximations can allow to optimize such hyperparameters just like standard neural network parameters using gradients and on the training data. However, estimating a single hyperparameter gradient requires a pass through the entire dataset, limiting the scalability of such algorithms. In this work, we overcome this issue by introducing lower bounds to the linearized Laplace approximation of the marginal likelihood. In contrast to previous estimators, these bounds are amenable to stochastic-gradient-based optimization and allow to trade off estimation accuracy against computational complexity. We derive them using the function-space form of the linearized Laplace, which can be estimated using the neural tangent kernel. Experimentally, we show that the estimators can significantly accelerate gradient-based hyperparameter optimization.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2306.03968")
get_code_for_paper("2306.03968")
have("2306.03968")
Connect an agent — have() is free.