We lifted 7 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| facebookresearch/qhoptim | canonical | 6 of 7 |
| Function | Status | Where it lives |
|---|---|---|
| allclose | Ran | facebookresearch/qhoptim/test_qhoptim/tf/util.py pointer only (licence: MIT) · get_code("5597447a89943bbe") |
| build_net | Ran | facebookresearch/qhoptim/test_qhoptim/tf/util.py pointer only (licence: MIT) · get_code("8c61c8f36e03fd81") |
| call_if_callable | Ran | facebookresearch/qhoptim/qhoptim/tf/util.py pointer only (licence: MIT) · get_code("f70286c6efb24e49") |
| from_pid | Ran | facebookresearch/qhoptim/qhoptim/common/param_conv.py pointer only (licence: MIT) · get_code("85d5bd9c96b637ab") |
| from_robust_momentum | Ran | facebookresearch/qhoptim/qhoptim/common/param_conv.py pointer only (licence: MIT) · get_code("f7a58c6038a0e8fa") |
| from_synthesized_nesterov | Ran | facebookresearch/qhoptim/qhoptim/common/param_conv.py pointer only (licence: MIT) · get_code("e8d42db83273329b") |
| QHAdamW | Not yet run | facebookresearch/qhoptim/qhoptim/pyt/qhadam.py pointer only (licence: MIT) · get_code("e6d8e5bc3f47df76") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Momentum-based acceleration of stochastic gradient descent (SGD) is widely used in deep learning. We propose the quasi-hyperbolic momentum algorithm (QHM) as an extremely simple alteration of momentum SGD, averaging a plain SGD step with a momentum step. We describe numerous connections to and identities with other algorithms, and we characterize the set of two-state optimization algorithms that QHM can recover. Finally, we propose a QH variant of Adam called QHAdam, and we empirically demonstrate that our algorithms lead to significantly improved training in a variety of settings, including a new state-of-the-art result on WMT16 EN-DE. We hope that these empirical results, combined with the conceptual and practical simplicity of QHM and QHAdam, will spur interest from both practitioners and researchers. Code is immediately available.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("1810.06801")
get_code_for_paper("1810.06801")
have("1810.06801")
Connect an agent — have() is free.