Rethinking Local Learning: A Cheaper and Faster Recipe for LLM Post-Training

Xu Yang, Junhao Su, Peizhe Wang, Tianyang Han, Hengyu Shi, Zhiling Wang

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 4 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• HumyuShi/LoPT

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

LLM post-training typically propagates task gradients through the full depth of the model. Although this end-to-end structure is simple and general, it couples task adaptation to full-depth activation storage, long-range backward dependencies and direct task-gradient access to pretrained representations. We argue that this full-depth backward coupling can be unnecessarily expensive and intrusive, particularly when post-training supervision is much narrower than pre-training. To this end, we propose LoPT: Local-Learning Post-Training, a simple post-training strategy that makes gradient reach an explicit design choice. LoPT places a single gradient boundary at the transformer midpoint: the second-half block learns from the task objective, while the first-half block is updated by a lightweight feature-reconstruction objective to preserve useful representations and maintain interface compatibility. LoPT shortens the task-induced backward path while limiting direct interference from narrow task gradients on early-layer representations. Extensive experiments demonstrate that LoPT achieves competitive performance with lower memory cost, higher training efficiency and better retention of pretrained capabilities. Our code is available at: https://github.com/HumyuShi/LoPT

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2605.04913")
get_code_for_paper("2605.04913")
have("2605.04913")

Connect an agent — have() is free.