We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
The rapid evolution of Multimodal Large Language Models (MLLMs) has brought substantial advancements in artificial intelligence, significantly enhancing the capability to understand and generate multimodal content. While prior studies have largely concentrated on model architectures and training methodologies, a thorough analysis of the benchmarks used for evaluating these models remains underexplored. This survey addresses this gap by systematically reviewing 211 benchmarks that assess MLLMs across four core domains: understanding, reasoning, generation, and application. We provide a detailed analysis of task designs, evaluation metrics, and dataset constructions, across diverse modalities. We hope that this survey will contribute to the ongoing advancement of MLLM research by offering a comprehensive overview of benchmarking practices and identifying promising directions for future work. An associated GitHub repository collecting the latest papers is available.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2409.18142")
get_code_for_paper("2409.18142")
have("2409.18142")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2409.18142.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2409.18142)
Connect an agent — have() is free.