SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2411.09734 · 2024

Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equations

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 1 functions out of this paper's own repositories and ran 0 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
carlosherediapimienta/numerical_simulations_nonlocal canonical 0 of 1
FunctionStatusWhere it lives
chunked_vmap Not yet run carlosherediapimienta/numerical_simulations_nonlocal/solvers/adagradsolver.py
code served (permissive licence) · get_code("f0c2daedaa2e12d7")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

In this paper, we propose a continuous-time formulation for the AdaGrad, RMSProp, and Adam optimization algorithms by modeling them as first-order integro-differential equations. We perform numerical simulations of these equations, along with stability and convergence analyses, to demonstrate their validity as accurate approximations of the original algorithms. Our results indicate a strong agreement between the behavior of the continuous-time models and the discrete implementations, thus providing a new perspective on the theoretical understanding of adaptive optimization methods.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2411.09734")
get_code_for_paper("2411.09734")
have("2411.09734")

Connect an agent — have() is free.