We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Pretrained models are the standard starting point for training. This approach consistently outperforms the use of a random initialization. However, pretraining is a costly endeavour that few can undertake. In this paper, we create better base models at hardly any cost, by fusing multiple existing fine tuned models into one. Specifically, we fuse by averaging the weights of these models. We show that the fused model results surpass the pretrained model ones. We also show that fusing is often better than intertraining. We find that fusing is less dependent on the target task. Furthermore, weight decay nullifies intertraining effects but not those of fusing.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2204.03044")
get_code_for_paper("2204.03044")
have("2204.03044")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2204.03044.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2204.03044)
Connect an agent — have() is free.