SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2505.23422 · 2025

From Knowledge to Noise: CTIM-Rover and the Pitfalls of Episodic Memory in Software Engineering Agents

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 3 functions out of this paper's own repositories and ran 1 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
liqs-v2/ctim-rover canonical 1 of 3
FunctionStatusWhere it lives
update_costs Ran liqs-v2/ctim-rover/ctim-rover-scripts/knowledge_distiller.py
pointer only (licence: NOASSERTION) · get_code("787766420c613ab4")
prompt_o1_model_with Not yet run liqs-v2/ctim-rover/ctim-rover-scripts/knowledge_distiller.py
pointer only (licence: NOASSERTION) · get_code("067a62c96c1c5542")
prompt_sonnet_37_model_with Not yet run liqs-v2/ctim-rover/ctim-rover-scripts/knowledge_distiller.py
pointer only (licence: NOASSERTION) · get_code("fd9fbbe2c68337a6")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We introduce CTIM-Rover, an AI agent for Software Engineering (SE) built on top of AutoCodeRover (Zhang et al., 2024) that extends agentic reasoning frameworks with an episodic memory, more specifically, a general and repository-level Cross-Task-Instance Memory (CTIM). While existing open-source SE agents mostly rely on ReAct (Yao et al., 2023b), Reflexion (Shinn et al., 2023), or Code-Act (Wang et al., 2024), all of these reasoning and planning frameworks inefficiently discard their long-term memory after a single task instance. As repository-level understanding is pivotal for identifying all locations requiring a patch for fixing a bug, we hypothesize that SE is particularly well positioned to benefit from CTIM. For this, we build on the Experiential Learning (EL) approach ExpeL (Zhao et al., 2024), proposing a Mixture-Of-Experts (MoEs) inspired approach to create both a general-purpose and repository-level CTIM. We find that CTIM-Rover does not outperform AutoCodeRover in any configuration and thus conclude that neither ExpeL nor DoT-Bank (Lingam et al., 2024) scale to real-world SE problems. Our analysis indicates noise introduced by distracting CTIM items or exemplar trajectories as the likely source of the performance degradation.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2505.23422")
get_code_for_paper("2505.23422")
have("2505.23422")

Connect an agent — have() is free.