Unleashing the Potential of Consistency Learning for Detecting and Grounding Multi-Modal Media Manipulation

Yang Yang, Xu Zhou, Yiheng Li, Huan Liu, Weihua Chen, Zichang Tan, Zhen Lei

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 24 functions out of this paper's own repositories and ran 13 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

Repositories linked to this paper

• liyih/CSCL

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

To tackle the threat of fake news, the task of detecting and grounding multi-modal media manipulation (DGM 4 ) has received increasing attention. However, most stateof-the-art methods fail to explore the fine-grained consistency within local content, usually resulting in an inadequate perception of detailed forgery and unreliable results. In this paper, we propose a novel approach named Contextual-Semantic Consistency Learning (CSCL) to enhance the fine-grained perception ability of forgery for DGM 4 . Two branches for image and text modalities are established, each of which contains two cascaded decoders, i.e., Contextual Consistency Decoder (CCD) and Semantic Consistency Decoder (SCD), to capture within-modality contextual consistency and across-modality semantic consistency, respectively. Both CCD and SCD adhere to the same criteria for capturing fine-grained forgery details. To be specific, each module first constructs consistency features by leveraging additional supervision from the heterogeneous information of each token pair. Then, the forgeryaware reasoning or aggregating is adopted to deeply seek forgery cues based on the consistency features. Extensive experiments on DGM 4 datasets prove that CSCL achieves new state-of-the-art performance, especially for the results of grounding manipulated content. Codes and weights are avaliable at https://github.com/liyih/CSCL.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2506.05890")
get_code_for_paper("2506.05890")
have("2506.05890")

Connect an agent — have() is free.