SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2502.13061 · EMNLP · 2025

Robust Adaptation of Large Multimodal Models for Retrieval Augmented Hateful Meme Detection

Guangyu Yang, Weizhe Lin, Jinghong Chen, Jingbiao Mei, Bill Byrne

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 4 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
JingbiaoMei/RGCL — 2 of 4
FunctionStatusWhere it lives
compute_l2 Ran JingbiaoMei/RGCL/src/model/loss.py
code served (permissive licence) · get_code("b79d1b8058e01331")
sparse_retrieve_hard_negatives_pseudo_positive Ran JingbiaoMei/RGCL/src/model/loss.py
code served (permissive licence) · get_code("a20178fd77f8b5a5")
compute_loss Not yet run JingbiaoMei/RGCL/src/model/loss.py
code served (permissive licence) · get_code("e29c3311f99491d1")
dense_retrieve_hard_negatives_pseudo_positive Not yet run JingbiaoMei/RGCL/src/model/loss.py
code served (permissive licence) · get_code("20d79017dbff7d0c")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Hateful memes have become a significant concern on the Internet, necessitating robust automated detection systems. While Large Multimodal Models (LMMs) have shown promise in hateful meme detection, they face notable challenges like sub-optimal performance and limited out-of-domain generalization capabilities. Recent studies further reveal the limitations of both supervised fine-tuning (SFT) and in-context learning when applied to LMMs in this setting. To address these issues, we propose a robust adaptation framework for hateful meme detection that enhances in-domain accuracy and cross-domain generalization while preserving the general vision-language capabilities of LMMs. Analysis reveals that our approach achieves improved robustness under adversarial attacks compared to SFT models. Experiments on six meme classification datasets show that our approach achieves state-of-theart performance, outperforming larger agentic systems. Moreover, our method generates higher-quality rationales for explaining hateful content compared to standard SFT, enhancing model interpretability. Code available at https://github.com/JingbiaoMei/RGCL This paper contains content for demonstration purposes that may be disturbing for some readers.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2502.13061")
get_code_for_paper("2502.13061")
have("2502.13061")

Connect an agent — have() is free.