SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2504.10141 · ICLR · 2025

The Impact of Model Zoo Size and Composition on Weight Space Learning

Damian Borth, Konstantin Sch Ürholt, Damian Falk

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 10 functions out of this paper's own repositories and ran 6 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
HSG-AIML/MultiZoo-SANE canonical 6 of 10
FunctionStatusWhere it lives
clean_checkpoint_keys Ran HSG-AIML/MultiZoo-SANE/src/mzsane/datasets/dataset_tokens.py
code served (permissive licence) · get_code("cadc8ba49c6865c0")
permute_model_vector Ran HSG-AIML/MultiZoo-SANE/src/mzsane/datasets/augmentations.py
code served (permissive licence) · get_code("889fe8497b0ba7c8")
read_properties_from_path Ran HSG-AIML/MultiZoo-SANE/src/mzsane/datasets/dataset_base.py
code served (permissive licence) · get_code("5ac745803cfc2796")
read_properties_from_path Ran HSG-AIML/MultiZoo-SANE/src/mzsane/datasets/dataset_epochs.py
code served (permissive licence) · get_code("3b4357d601080776")
test_checkpoint_for_nan Ran HSG-AIML/MultiZoo-SANE/src/mzsane/datasets/dataset_auxiliaries.py
code served (permissive licence) · get_code("ec1ff9919f412754")
test_checkpoint_with_threshold Ran HSG-AIML/MultiZoo-SANE/src/mzsane/datasets/dataset_auxiliaries.py
code served (permissive licence) · get_code("d2a9bd8273d8f5d4")
get_net_epoch_lst_from_label Not yet run HSG-AIML/MultiZoo-SANE/src/mzsane/datasets/dataset_auxiliaries.py
code served (permissive licence) · get_code("31ce1adb9a55256c")
load_checkpoint Not yet run HSG-AIML/MultiZoo-SANE/src/mzsane/datasets/dataset_base.py
code served (permissive licence) · get_code("336c00ba04cc94cf")
load_checkpoints_remote Not yet run HSG-AIML/MultiZoo-SANE/src/mzsane/datasets/dataset_base.py
code served (permissive licence) · get_code("5fb96d161e21926b")
read_properties_from_path Not yet run HSG-AIML/MultiZoo-SANE/src/mzsane/datasets/dataset_properties.py
code served (permissive licence) · get_code("790ee3cda94b46c4")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Re-using trained neural network models is a common strategy to reduce training cost and transfer knowledge. Weight space learning -using the weights of trained models as data modality -is a promising new field to re-use populations of pre-trained models for future tasks. Approaches in this field have demonstrated high performance both on model analysis and weight generation tasks. However, until now their learning setup requires homogeneous model zoos where all models share the same exact architecture, limiting their capability to generalize beyond the population of models they saw during training. In this work, we remove this constraint and propose a modification to a common weight space learning method to accommodate training on heterogeneous populations of models. We further investigate the resulting impact of model diversity on generating unseen neural network model weights for zero-shot knowledge transfer. Our extensive experimental evaluation shows that including models with varying underlying image datasets has a high impact on performance and generalization, for both in-and out-of-distribution settings. Code is available on github.com/HSG-AIML/MultiZoo-SANE. Recently, Weight Space Learning has emerged as an additional approach to re-use populations of pre-trained models (

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2504.10141")
get_code_for_paper("2504.10141")
have("2504.10141")

Connect an agent — have() is free.