SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2412.04292 · 2024

SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Model

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 3 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
hzlsaber/sida canonical 0 of 1
copy not recorded — 2 of 2
FunctionStatusWhere it lives
dice_loss Ran this paper's copy was not recorded; identical code first harvested from maverickren/pixellm
pointer only · get_code("a1c2083ec2eaeb22")
sigmoid_ce_loss Ran this paper's copy was not recorded; identical code first harvested from maverickren/pixellm
pointer only · get_code("a9292f5d89194794")
parse_args Not yet run hzlsaber/sida/train_SIDA.py
pointer only (licence: NONE) · get_code("e1f0c716ee9c2cdf")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

The rapid advancement of generative models in creating highly realistic images poses substantial risks for misinformation dissemination. For instance, a synthetic image, when shared on social media, can mislead extensive audiences and erode trust in digital content, resulting in severe repercussions. Despite some progress, academia has not yet created a large and diversified deepfake detection dataset for social media, nor has it devised an effective solution to address this issue. In this paper, we introduce the Social media Image Detection dataSet (SID-Set), which offers three key advantages: (1) extensive volume, featuring 300K AI-generated/tampered and authentic images with comprehensive annotations, (2) broad diversity, encompassing fully synthetic and tampered images across various classes, and (3) elevated realism, with images that are predominantly indistinguishable from genuine ones through mere visual inspection. Furthermore, leveraging the exceptional capabilities of large multimodal models, we propose a new image deepfake detection, localization, and explanation framework, named SIDA (Social media Image Detection, localization, and explanation Assistant). SIDA not only discerns the authenticity of images, but also delineates tampered regions through mask prediction and provides textual explanations of the model's judgment criteria. Compared with state-of-the-art deepfake detection models on SID-Set and other benchmarks, extensive experiments demonstrate that SIDA achieves superior performance among diversified settings. The code, model, and dataset will be released.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2412.04292")
get_code_for_paper("2412.04292")
have("2412.04292")

Connect an agent — have() is free.