Assaf Schuster, Hadar Sivan, Moshe Gabel
We lifted 1 functions out of this paper's own repositories and ran 0 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| hsivan/fosi | canonical | 0 of 1 |
| Function | Status | Where it lives |
|---|---|---|
| loss_fn | Not yet run | hsivan/fosi/examples/fosi_torch_example.py code served (permissive licence) · get_code("a0b5a0ec8b3915d2") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Popular machine learning approaches forgo second-order information due to the difficulty of computing curvature in high dimensions. We present FOSI, a novel meta-algorithm that improves the performance of any base first-order optimizer by efficiently incorporating second-order information during the optimization process. In each iteration, FOSI implicitly splits the function into two quadratic functions defined on orthogonal subspaces, then uses a second-order method to minimize the first, and the base optimizer to minimize the other. We formally analyze FOSI's convergence and the conditions under which it improves a base optimizer. Our empirical evaluation demonstrates that FOSI improves the convergence rate and optimization time of first-order methods such as Heavy-Ball and Adam, and outperforms second-order methods (K-FAC and L-BFGS).
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2302.08484")
get_code_for_paper("2302.08484")
have("2302.08484")
Connect an agent — have() is free.