We lifted 3 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| r-three/realistic_evaluation_of_model_merging_for_compositional_generalization | canonical | 2 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| get_statisticsDir | Ran | r-three/realistic_evaluation_of_model_merging_for_compositional_generalization/src/merging/save_statistic.py pointer only (licence: NONE) · get_code("9fbff4f411c485ac") |
| topk_values_mask | Ran | r-three/realistic_evaluation_of_model_merging_for_compositional_generalization/src/merging/save_statistic.py pointer only (licence: NONE) · get_code("225c26ebc567caaa") |
| compute_taskVectors | Not yet run | r-three/realistic_evaluation_of_model_merging_for_compositional_generalization/src/merging/save_statistic.py pointer only (licence: NONE) · get_code("db20c6a9f6d0a3cd") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Merging has become a widespread way to cheaply combine individual models into a single model that inherits their capabilities and attains better performance. This popularity has spurred rapid development of many new merging methods, which are typically validated in disparate experimental settings and frequently differ in the assumptions made about model architecture, data availability, and computational budget. In this work, we characterize the relative merits of different merging methods by evaluating them in a shared experimental setting and precisely identifying the practical requirements of each method. Specifically, our setting focuses on using merging for compositional generalization of capabilities in image classification, image generation, and natural language processing. Additionally, we measure the computational costs of different merging methods as well as how they perform when scaling the number of models being merged. Taken together, our results clarify the state of the field of model merging and provide a comprehensive and rigorous experimental setup to test new methods.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2409.18314")
get_code_for_paper("2409.18314")
have("2409.18314")
Connect an agent — have() is free.