Substrate Asymmetry in User-Side Memory: A Diagnostic Framework

Youwang Deng

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 26 functions out of this paper's own repositories and ran 15 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• EpistemicaLab/substrate-asymmetry-memory

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

User-side memory in LLMs is typically scored as a single "personalization" capability -given a user's history, is the output more user-aware? We show this aggregate metric hides opposite-direction failures. Memory factorises into at least three orthogonal axes -behavioral consistency (style, voice), factual presence (recall facts in the history), and factual absence (abstain when a fact is absent) -and no single substrate wins all three. Comparing per-user γ-LoRA (a small LoRA adapter trained on each user's history; γ denotes per-user, not per-task) against BGE-large dense top-K retrieval on a controlled 50-user synthetic corpus and a real-data probe (Salemi et al., 2024), we find γ-LoRA decisively wins behavioral style while RAG decisively wins factual absence -and the same query-projection cells in attention layers 21-35 causally load-bear both effects in opposite directions (zeroing those LoRA weights raises absenceprobe true-positive rate by +33 percentage points and drops presence-probe TPR by 20 pp). On the more heavily RLHF-tuned Llama-3.1-8B-Instruct the asymmetry strengthens, not heals: parametric memory's behavioral advantage collapses while its absence-calibration deficit against retrieval widens -an alignment tax on parametric user-memory. On real-data LaMP-3, γ-LoRA underperforms a majority baseline; a 9-condition mitigation sweep diagnoses this as instruction-following collapse, not substrate failure (a 9 × 2 mitigation crossproduct shows the eval-time {1..5} logit mask drives main_acc to ≥0.995 on every training recipe -recipe choice does not escape the collapse, and the residual probe2 ceiling 0.605-0.660 is task-structural, not recipe-tunable), and the best training-time fix replicates bit-identically on Llama. Finally, substrate-selection routing is question-classification, not calibration: a 110M DistilBERT on the question text alone beats every logit-based router. We contribute the diagnostic framework, the diagnosed real-data negative, the alignment-tax replication, and the routing-as-classification finding.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2606.11712")
get_code_for_paper("2606.11712")
have("2606.11712")

Connect an agent — have() is free.