KnowledgeSmith: Uncovering Knowledge Updating in LLMs with Model Editing and Unlearning

Jindong Wang, Hao Chen, Marios Savvides, Sharon Li, Yinyi Luo, Zhexian Zhou, Kai Qiu

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 11 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• AIFrontierLab/KnowledgeSmith

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Knowledge editing and machine unlearning are two popular approaches for large language models (LLMs) to stay up-to-date. However, the knowledge updating mechanism of LLMs remains largely unexplored due to insufficient, isolated, and small-scale evaluation. For instance, are LLMs similar to humans in modifying certain knowledge? What differs editing and unlearning as training data increases? This paper proposes KnowledgeSmith, a unified framework to systematically understand the updating mechanism of LLMs. We first cast editing and unlearning as instances of one constrained optimization problem. Then, we propose an automatic dataset generator that provides structured interventions across multiple graph levels and data scales, enabling controlled studies of how different modification strategies propagate through model knowledge. Extensive experiments demonstrate nuanced insights over knowledge propagation, plasticity scaling, consistency, and robustness. For instance, our results show that LLMs do not exhibit similar updating as humans for different levels of knowledge, and there exists consistency-capacity trade-off. We hope our findings can offer suggestions to the design of more reliable and scalable strategies. Code: https://github.com/AIFrontierLab/KnowledgeSmith.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2510.02392")
get_code_for_paper("2510.02392")
have("2510.02392")

Connect an agent — have() is free.