Yiyi Zhou, Qiong Wu, Wei Yu, Yutao Jiang, Wenhao Lin
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Recent Multimodal Large Language Models (MLLMs) often use a large number of visual tokens to compensate their visual shortcoming, leading to excessive computation and obvious visual redundancy. In this paper, we investigate what kind of visual tokens are needed for MLLMs, and reveal that both foreground and background tokens are critical for MLLMs given the varying difficulties of examples. Based on this observation, we propose a graph-based method towards training-free visual token pruning, termed G-Prune. In particular, G-Prune regards visual tokens as nodes, and construct their connections based on their semantic similarities. Afterwards, the information flow is propagated via weighted links, and the most important tokens after iterations are kept for MLLMs, which can be front or background. To validate G-Prune, we apply it to a recent MLLM called LLaVA-NeXT, and conduct extensive experiments on a set of benchmarks. The experiment results show that G-Prune can greatly reduce computation overhead while retaining high performance on both coarse-and fine-grained tasks. For instance, G-Prune can reduce 63.57% FLOPs of LLaVA-NeXT on VQA2.0 and TextVQA with only 0.95% and 2.34% accuracy drops, respectively.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2501.02268")
get_code_for_paper("2501.02268")
have("2501.02268")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2501.02268.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2501.02268)
Connect an agent — have() is free.