SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 1605.04131 · 2016

Barzilai-Borwein Step Size for Stochastic Gradient Descent

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 7 functions out of this paper's own repositories and ran 0 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
SyneRBI/PETRIC-MaGeZ pwc_unofficial 0 of 7
FunctionStatusWhere it lives
load_interfile_image Not yet run SyneRBI/PETRIC-MaGeZ/plot_data.py
code served (permissive licence) · get_code("4a191a51c146b95d")
load_interfile_image Not yet run SyneRBI/PETRIC-MaGeZ/simulation_src/fig7_fig_8_challenge.py
code served (permissive licence) · get_code("2587457df9ca3fe1")
neighbor_difference_and_sum Not yet run SyneRBI/PETRIC-MaGeZ/rdp.py
code served (permissive licence) · get_code("31f80f211d49b55e")
neighbor_offsets Not yet run SyneRBI/PETRIC-MaGeZ/rdp.py
code served (permissive licence) · get_code("e7134a80c918c0a7")
neighbor_product Not yet run SyneRBI/PETRIC-MaGeZ/rdp.py
code served (permissive licence) · get_code("82e3d53f70395607")
parse_interfile_header Not yet run SyneRBI/PETRIC-MaGeZ/plot_data.py
code served (permissive licence) · get_code("a0ebd0be7d6e9f8f")
scalars Not yet run SyneRBI/PETRIC-MaGeZ/simulation_src/fig7_fig_8_challenge.py
code served (permissive licence) · get_code("7962b724006daa74")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

One of the major issues in stochastic gradient descent (SGD) methods is how to choose an appropriate step size while running the algorithm. Since the traditional line search technique does not apply for stochastic optimization algorithms, the common practice in SGD is either to use a diminishing step size, or to tune a fixed step size by hand, which can be time consuming in practice. In this paper, we propose to use the Barzilai-Borwein (BB) method to automatically compute step sizes for SGD and its variant: stochastic variance reduced gradient (SVRG) method, which leads to two algorithms: SGD-BB and SVRG-BB. We prove that SVRG-BB converges linearly for strongly convex objective functions. As a by-product, we prove the linear convergence result of SVRG with Option I proposed in [10], whose convergence result is missing in the literature. Numerical experiments on standard data sets show that the performance of SGD-BB and SVRG-BB is comparable to and sometimes even better than SGD and SVRG with best-tuned step sizes, and is superior to some advanced SGD variants.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("1605.04131")
get_code_for_paper("1605.04131")
have("1605.04131")

Connect an agent — have() is free.