Reasoning or Retrieval? A Study of Answer Attribution on Large Reasoning Models

Jiacheng Liang, Yuhui Wang, Changjiang Li, Ting Wang, Guangke Chen

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 19 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• ZJUWYH/FARL

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Large reasoning models (LRMs) exhibit unprecedented capabilities in solving complex problems through Chain-of-Thought (CoT) reasoning. However, recent studies reveal that their final answers often contradict their own reasoning traces. We hypothesize that this inconsistency stems from two competing mechanisms for generating answers: CoT reasoning and memory retrieval. To test this hypothesis, we conduct controlled experiments that challenge LRMs with misleading cues during reasoning and/or corrupted answers during retrieval. Our results across models and datasets confirm that both mechanisms operate simultaneously, with their relative dominance influenced by multiple factors: problem domains, model scales, and fine-tuning approaches (e.g., reinforcement learning vs. distillation). The findings reveal a critical limitation in current reasoning fine-tuning paradigms: models can exploit the retrieval mechanism as a shortcut, effectively "hacking" the reward signal and undermining genuine reasoning development. To address this challenge, we introduce FARL, 1 a novel fine-tuning framework that integrates memory unlearning with reinforcement learning. By carefully suppressing retrieval shortcuts during the fine-tuning process, FARL promotes reasoningdominant behavior and enhances generalizable reasoning capabilities. The code is available: https://github.com/ZJUWYH/FARL

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2509.24156")
get_code_for_paper("2509.24156")
have("2509.24156")

Connect an agent — have() is free.