SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2510.06888 · EMNLP · 2025

M3Retrieve: Benchmarking Multimodal Retrieval for Medicine

Sriparna Saha, Pradeepika Verma, Akash Ghosh, Arkadeep Acharya, Kitsuchart Pasupa, Priti Singh

arXiv · PDF · Open in the Atlas

Code that ran

We lifted 1 functions out of this paper's own repositories and ran 0 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.

RepositoryRoleRan
AkashGhosh/M3Retrieve canonical 0 of 1
FunctionStatusWhere it lives
concat_all_gather Not yet run AkashGhosh/M3Retrieve/Code/blip_ff.py
pointer only (licence: NONE) · get_code("73cecca9f3575f09")

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

With the increasing use of Retrieval-Augmented Generation (RAG), strong retrieval models have become more important than ever. In healthcare, multimodal retrieval models that combine information from both text and images offer major advantages for many downstream tasks such as question answering, cross-modal retrieval, and multimodal summarization, since medical data often includes both formats. However, there is currently no standard benchmark to evaluate how well these models perform in medical settings. To address this gap, we introduce M3Retrieve, a Multimodal Medical Retrieval Benchmark. M3Retrieve, spans 5 domains,16 medical fields, and 4 distinct tasks, with over 1.2 Million text documents and 164K multimodal queries, all collected under approved licenses. We evaluate leading multimodal retrieval models on this benchmark to explore the challenges specific to different medical specialities and to understand their impact on retrieval performance. By releasing M3Retrieve, we aim to enable systematic evaluation, foster model innovation, and accelerate research toward building more capable and reliable multimodal retrieval systems for medical applications. The dataset and the baselines code are available in this github page https: //github.com/AkashGhosh/M3Retrieve.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2510.06888")
get_code_for_paper("2510.06888")
have("2510.06888")

Connect an agent — have() is free.