SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2603.01526 · 2026

Scalable Multi-Task Low-Rank Model Adaptation

Qianru Sun, Antoine Ledent, Zichen Tian

arXiv · PDF · Open in the Atlas

Code that ran

7 of 12 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (6 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 0 violated it and 6 have no contract verdict; 1 failure of our instrument; 5 no recorded run), measured 2026-11-22; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2603.01526.json · badge: /api/ran/2603.01526.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
doem97/ICLR26_mtLoRA community
Papers with Code archive
7 ran · 5 no recorded run
FunctionStatusWhere it lives
Expert Ran
ran on a synthesized input · constructed an object; no result was computed
doem97/ICLR26_mtLoRA/peft/tuners/mmoelora.py
pointer only (licence: NONE) · get_code("ed21676126691b8d")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, doem97/ICLR26_mtLoRA when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Gate Ran
ran on a synthesized input · constructed an object; no result was computed
doem97/ICLR26_mtLoRA/peft/tuners/mmoelora.py
pointer only (licence: NONE) · get_code("06b0bde3f28af133")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, doem97/ICLR26_mtLoRA when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

LoraLinear Ran
ran on a synthesized input · constructed an object; no result was computed
doem97/ICLR26_mtLoRA/peft/tuners/mmoelora.py
pointer only (licence: NONE) · get_code("020556a5f9913cff")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, doem97/ICLR26_mtLoRA when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

MMOELinearA Ran
ran on a synthesized input · constructed an object; no result was computed
doem97/ICLR26_mtLoRA/peft/tuners/mmoelora.py
pointer only (licence: NONE) · get_code("f19dba7d1405f0f3")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, doem97/ICLR26_mtLoRA when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

MMOELinearB Ran
ran on a synthesized input · constructed an object; no result was computed
doem97/ICLR26_mtLoRA/peft/tuners/mmoelora.py
pointer only (licence: NONE) · get_code("7f92c604b6065362")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, doem97/ICLR26_mtLoRA when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

MMOELoraLayer Ran
ran on a synthesized input · constructed an object; no result was computed
doem97/ICLR26_mtLoRA/peft/tuners/mmoelora.py
pointer only (licence: NONE) · get_code("d68ee8a009b9600b")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, doem97/ICLR26_mtLoRA when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

_get_submodules Ran
ran; our contract draft was wrong, not the code
doem97/ICLR26_mtLoRA/peft/tuners/mmoelora.py
pointer only (licence: NONE) · get_code("be205a37f47decee")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, doem97/ICLR26_mtLoRA when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

LoraModel Not yet run
unverified (harvested, no recorded run)
doem97/ICLR26_mtLoRA/peft/tuners/mmoelora.py
pointer only (licence: NONE) · get_code("7fa29f6b1cc50dc7")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, doem97/ICLR26_mtLoRA when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

MMOELoraLinear Not yet run
unverified (harvested, no recorded run)
doem97/ICLR26_mtLoRA/peft/tuners/mmoelora.py
pointer only (licence: NONE) · get_code("08b7be0cda0022da")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, doem97/ICLR26_mtLoRA when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

MMOELoraModel Not yet run
unverified (harvested, no recorded run)
doem97/ICLR26_mtLoRA/peft/tuners/mmoelora.py
pointer only (licence: NONE) · get_code("0c8364a1277289a6")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, doem97/ICLR26_mtLoRA when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

_freeze_adapter Not yet run
unverified (harvested, no recorded run)
doem97/ICLR26_mtLoRA/peft/tuners/mmoelora.py
pointer only (licence: NONE) · get_code("fe8f6b176139f916")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, doem97/ICLR26_mtLoRA when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

mark_only_lora_as_trainable Not yet run
unverified (harvested, no recorded run)
doem97/ICLR26_mtLoRA/peft/tuners/mmoelora.py
pointer only (licence: NONE) · get_code("0e7c780ef1f1ff8b")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, doem97/ICLR26_mtLoRA when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

Also present in other papers' code

Of 12 samples lifted from this paper's linked repositories, 12 carry a function identity and 0 do not. Of the 12 distinct functions with an identity, 2 are also present in other papers whose code we hold (2 with at least one paper dated earlier, 0 the same month, 1 later) and 10 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
_get_submodules
brick of this paper's assembly: helper
5 other papers
4 dated earlier, 1 later · 4 only as an assembly brick
2106.09685 (2021-06)
LoRA: Low-Rank Adaptation of Large Language Models
2402.13717 (2024-02)
Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent
2408.10159 (2024-08)
Customizing Language Models with Instance-wise LoRA for Sequential Recommendation
and 2 more not named here
mark_only_lora_as_trainable
brick of this paper's assembly: helper
3 other papers
3 dated earlier · 1 only as an assembly brick
2106.09685 (2021-06)
LoRA: Low-Rank Adaptation of Large Language Models
2503.12897 (2025-03)
Federated Continual Instruction Tuning
2507.02302 (2025-07)
DoMIX: An Efficient Framework for Exploiting Domain Knowledge in Fine-Tuning

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Scaling multi-task low-rank adaptation (LoRA) to a large number of tasks induces catastrophic performance degradation, such as an accuracy drop from 88.2% to 2.0% on DOTA when scaling from 5 to 15 tasks. This failure is due to parameter and representation misalignment. We find that existing solutions, like regularization and dynamic routing, fail at scale because they are constrained by a fundamental trade-off: strengthening regularization to reduce inter-task conflict inadvertently suppresses the essential feature discrimination required for effective routing. In this work, we identify two root causes for this trade-off. First, uniform regularization disrupts inter-task knowledge sharing: shared underlying knowledge concentrates in high-SV components (89% alignment on Flanv2→BBH). Uniform regularization forces high-SV components to update in orthogonal directions, directly disrupting the shared knowledge. Second, Conflict Amplification: Applying LoRA at the component-level (e.g., W q , W v ) amplifies gradient conflicts; we show block-level adaptation reduces this conflict by 76% with only 50% parameters. Based on these insights, we propose mtLoRA, a scalable solution with three novel designs: 1) Spectral-Aware Regularization to selectively orthogonalize low-SV components while preserving high-SV shared knowledge, 2) Block-Level Adaptation to mitigate conflict amplification and largely improve parameter efficiency, and 3) Fine-Grained Routing using dimension-specific weights for superior expressive power. On four large-scale (15-25 tasks) vision (DOTA and iNat2018) and NLP (Dolly-15k and BBH) benchmarks, mtLoRA achieves 91.7%, 81.5%, 44.5% and 38.5% accuracy on DOTA, iNat2018, Dolly-15k and BBH respectively, outperforming the state-of-the-art by 2.3% on average while using 47% fewer parameters and 24% less training time. Code is available at https://github.com/doem97/ICLR26_mtLoRA.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2603.01526")
get_code_for_paper("2603.01526")
have("2603.01526")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2603.01526.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2603.01526.svg)](https://syntology.ai/paper/2603.01526)

Connect an agent — have() is free.