SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 1709.06709 · 2017

Online Learning of a Memory for Learning Rates

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 2 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
fmeier/online-meta-learning canonical 2 of 2
FunctionStatusWhere it lives
compute_class_rate Ran fmeier/online-meta-learning/meta_learning/backend/pytorch/learning.py
code served (permissive licence) · get_code("2b7dec8e2f8e9086")
compute_test_class_rate Ran fmeier/online-meta-learning/meta_learning/backend/pytorch/learning.py
code served (permissive licence) · get_code("76b494bf18006d4f")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

The promise of learning to learn for robotics rests on the hope that by extracting some information about the learning process itself we can speed up subsequent similar learning tasks. Here, we introduce a computationally efficient online meta-learning algorithm that builds and optimizes a memory model of the optimal learning rate landscape from previously observed gradient behaviors. While performing task specific optimization, this memory of learning rates predicts how to scale currently observed gradients. After applying the gradient scaling our meta-learner updates its internal memory based on the observed effect its prediction had. Our meta-learner can be combined with any gradient-based optimizer, learns on the fly and can be transferred to new optimization tasks. In our evaluations we show that our meta-learning algorithm speeds up learning of MNIST classification and a variety of learning control tasks, either in batch or online learning settings.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("1709.06709")
get_code_for_paper("1709.06709")
have("1709.06709")

Connect an agent — have() is free.