SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2403.10929 · 2024

Function-space Parameterization of Neural Networks for Sequential Learning

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 1 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
AaltoML/sfr-experiments canonical 1 of 1
FunctionStatusWhere it lives
kernel_fn Ran AaltoML/sfr-experiments/experiments/cl/models/sfr.py
code served (permissive licence) · get_code("0ccf872967d99a8d")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Sequential learning paradigms pose challenges for gradient-based deep learning due to difficulties incorporating new data and retaining prior knowledge. While Gaussian processes elegantly tackle these problems, they struggle with scalability and handling rich inputs, such as images. To address these issues, we introduce a technique that converts neural networks from weight space to function space, through a dual parameterization. Our parameterization offers: (i) a way to scale function-space methods to large data sets via sparsification, (ii) retention of prior knowledge when access to past data is limited, and (iii) a mechanism to incorporate new data without retraining. Our experiments demonstrate that we can retain knowledge in continual learning and incorporate new data efficiently. We further show its strengths in uncertainty quantification and guiding exploration in model-based RL. Further information and code is available on the project website.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2403.10929")
get_code_for_paper("2403.10929")
have("2403.10929")

Connect an agent — have() is free.