SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2412.04467 · 2024

VisionZip: Longer is Better but Not Necessary in Vision Language Models

arXiv · PDF · Open in the Atlas

Code that ran

8 of 11 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 1 honoured the contract we drafted, 0 violated it and 0 have no contract verdict; 7 failures of our instrument; 3 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2412.04467.json · badge: /api/ran/2412.04467.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
dvlab-research/visionzip canonical (classifier label)
classifier basis: census
8 ran · 3 no recorded run
FunctionStatusWhere it lives
CLIP_EncoderLayer_forward Ran
ran; our contract draft was wrong, not the code
dvlab-research/visionzip/visionzip/utils.py
code served (permissive licence) · get_code("27a169cf3a211ef5")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from typing import Optional, Tuple

torch.manual_seed(0)

# Build a minimal self-attention module that matches the interface
class MockSelfAttention(nn.Module):
    def __init__(self, embed_dim, num_heads):
        super().__init__()
        self.embed_dim = embed_dim
        self.num_heads = num_heads
    
    def forward(self, hidden_states, attention_mask, causal_attention_mask, output_attentions=False):
        # Return (hidden_states, attn_weights, metric)
        batch_size, seq_len, embed_dim = hidden_states.shape
        attn_weights = torch.randn(batch_size, self.num_heads, seq_len, seq_len)
        metric = torch.tensor(0.5)
        return hidden_states, attn_weights, metric

# Build a minimal MLP module
class MockMLP(nn.Module):
    def __init__(self, embed_dim):
        super().__init__()
        self.linear = nn.Linear(embed_dim, embed_dim)
    
    def forward(self, hidden_states):
        return self.linear(hidden_states)

# Build the encoder layer
class MockCLIPEncoderLayer(nn.Module):
    def __init__(self, embed_dim, num_heads):
        super().__init__()
        self.layer_norm1 = nn.LayerNorm(embed_dim)
        self.layer_norm2 = nn.LayerNorm(embed_dim)
        self.self_attn = MockSelfAttention(embed_dim, num_heads)
        self.mlp = MockMLP(embed_dim)
        self._info = {'r': [1, 0]}  # Pop values from this list
    
    def forward(self, hidden_states, attention_mask, causal_attention_mask, output_attentions=False):
        residual = hidden_states
        hidden_states = self.layer_norm1(hidden_states)
        hidden_states, attn_weights, metric = self.self_attn(
            hidden_states=hidden_states,
            attention_mask=attention_mask,
            causal_attention_mask=causal_attention_mask,
            output_attentions=output_attentions
        )
        hidden_states = residual + hidden_states
        r = self._info['r'].pop(0)
        if r > 0:
            self.metric = metric
        residual = hidden_states
        hidden_states = self.layer_norm2(hidden_states)
        hidden_states = self.mlp(hidden_states)
        hidden_states = residual + hidden_states
        outputs = (hidden_states,)
        if output_attentions:
            outputs += (attn_weights,)
        return outputs

# Set up parameters
batch_size = 2
seq_len = 5
embed_dim = 4
num_heads = 2

encoder_layer = MockCLIPEncoderLayer(embed_dim, num_heads)

hidden_states = torch.randn(batch_size, seq_len, embed_dim)
attention_mask = torch.zeros(batch_size, 1, seq_len, seq_len)
causal_attention_mask = torch.zeros(batch_size, 1, seq_len, seq_len)

ARGS = {
    'self': encoder_layer,
    'hidden_states': hidden_states,
    'attention_mask': attention_mask,
    'causal_attention_mask': causal_attention_mask,
    'output_attentions': False
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
encode_images_visionzip Ran
ran; our contract draft was wrong, not the code
dvlab-research/visionzip/visionzip/llava_arch.py
code served (permissive licence) · get_code("1585cd5e445a26b9")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 3, dtype family float, shape (2, 8, 5) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

class VisionTower(nn.Module):
    def __init__(self):
        super().__init__()
    
    def forward(self, images):
        # Returns image_features and keep_idx
        batch_size = images.shape[0]
        image_features = torch.randn(batch_size, 8, 4)  # (batch=2, seq=8, hidden=4)
        keep_idx = torch.arange(batch_size)
        return image_features, keep_idx

class MMProjector(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(4, 5)
    
    def forward(self, x):
        return self.linear(x)

class MockModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.vision_tower = VisionTower()
        self.mm_projector = MMProjector()
    
    def get_vision_tower(self):
        return self.vision_tower

class MockSelf:
    def __init__(self):
        self.model = MockModel()
    
    def get_model(self):
        return self.model

# Create instance and inputs
self_instance = MockSelf()
images = torch.randn(2, 3, 224, 224)  # (batch=2, channels=3, height=224, width=224)

ARGS = {
    "self": self_instance,
    "images": images
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 3, dtype family float, shape (2, 8, 5) — shape (2, 8, 5) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
encode_images_visionzip_multi Ran
ran; our contract draft was wrong, not the code
dvlab-research/visionzip/visionzip/llava_arch.py
code served (permissive licence) · get_code("e8c6fd834f76c9b1")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

class MockVisionTower(nn.Module):
    def __init__(self):
        super().__init__()
    
    def forward(self, images):
        # Returns (image_features, keep_idx)
        # image_features shape: (batch_size, num_patches, feature_dim)
        batch_size = images.shape[0]
        num_patches = 4
        feature_dim = 5
        image_features = torch.randn(batch_size, num_patches, feature_dim)
        keep_idx = torch.tensor([0, 1, 2, 3], dtype=torch.long)
        return image_features, keep_idx

class MockModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.vision_tower = MockVisionTower()
        self.mm_projector = nn.Linear(5, 7)  # Projects from 5 to 7 dims
    
    def get_vision_tower(self):
        return self.vision_tower

class MockSelf:
    def __init__(self):
        self.model = MockModel()
    
    def get_model(self):
        return self.model
    
    def encode_images_visionzip_multi(self, images):
        image_features, keep_idx = self.get_model().get_vision_tower().forward(images)
        image_features = self.get_model().mm_projector(image_features)
        return (image_features, keep_idx)

# Create mock self instance
mock_self = MockSelf()

# Create images tensor: (batch_size=2, channels=3, height=4, width=4)
images = torch.randn(2, 3, 4, 4)

ARGS = {
    "self": mock_self,
    "images": images
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
load_image Ran
ran; our contract draft was wrong, not the code
dvlab-research/visionzip/gradio_demo.py
code served (permissive licence) · get_code("7db2c78b1f3cc693")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Image. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

from PIL import Image
import numpy as np

# Create a small PIL Image object with RGB-like data
# The function calls .convert('RGB') on it, so we need a PIL Image
img_array = np.random.randint(0, 256, (4, 3, 3), dtype=np.uint8)
pil_image = Image.fromarray(img_array, mode='RGB')

ARGS = {
    'image': pil_image
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
PIL, numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Image — a Image output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a Image output the recorded ndim 3 and dtype family int describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
parse_r Ran
ran, honoured the contract we drafted
dvlab-research/visionzip/visionzip/utils.py
code served (permissive licence) · get_code("a54840c2e71e73aa")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from typing import List, Tuple, Union

def parse_r(num_layers: int, r: Union[List[int], Tuple[int, float], int]) -> List[int]:
    """
    Copy from the TOME. 
    https://github.com/facebookresearch/ToMe

    Process a constant r or r schedule into a list for use internally.

    r can take the following forms:
     - int: A constant number of tokens per layer.
     - Tuple[int, float]: A pair of r, inflection.
       Inflection describes there the the reduction / layer should trend
       upward (+1), downward (-1), or stay constant (0). A value of (r, 0)
       is as providing a constant r. (r, -1) is what we describe in the paper
       as "decreasing schedule". Any value between -1 and +1 is accepted.
     - List[int]: A specific number of tokens per layer. For extreme granularity.
    """
    inflect = 0
    if isinstance(r, list):
        if len(r) < num_layers:
            r = r + [0] * (num_layers - len(r))
        return list(r)
    elif isinstance(r, tuple):
        r, inflect = r
    min_val = int(r * (1.0 - inflect))
    max_val = 2 * r - min_val
    step = (max_val - min_val) / (num_layers - 1)
    return [int(min_val + step * i) for i in range(num_layers)]

ARGS = {
    "num_layers": 12,
    "r": 4
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family int — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
resize_image Ran
ran; our contract draft was wrong, not the code
dvlab-research/visionzip/gradio_demo.py
code served (permissive licence) · get_code("f170144b6e9abcec")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

from PIL import Image

# Create a small test image
image = Image.new('RGB', (100, 80), color='red')

# Define target size for resizing
target_size = (50, 40)

ARGS = {
    'image': image,
    'target_size': target_size
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
PIL — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
restore_image_features_sorted Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
dvlab-research/visionzip/visionzip/llava_arch.py
code served (permissive licence) · get_code("cbc15a0bfa5bc165")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (1154, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Create a minimal model class with the required attributes
class MinimalModel(nn.Module):
    def __init__(self, feature_dim):
        super().__init__()
        # image_newline is used in the function and needs to be a tensor
        self.image_newline = nn.Parameter(torch.randn(1, 1, feature_dim))
    
    def forward(self, x):
        return x

# Create a minimal wrapper class that mimics self in the function
class MinimalSelf:
    def __init__(self):
        self.model = MinimalModel(feature_dim=4)
    
    def restore_image_features_sorted(self, image_feature, cur_keep_idx, width, height):
        # This is the actual function body
        num_img, total_patches, feature_dim = image_feature.shape
        num_keep = cur_keep_idx.shape[1]
        num_extra = total_patches - num_keep
        cur_keep_idx_sorted, _ = cur_keep_idx.sort(dim=1)
        cur_keep_idx_sorted_restore = cur_keep_idx_sorted[:, 1:] - 1
        restored_features = torch.zeros((num_img, 576, feature_dim), device=image_feature.device, dtype=image_feature.dtype)
        mask = torch.zeros(num_img, 576, dtype=torch.bool, device=image_feature.device)
        mask.scatter_(1, cur_keep_idx_sorted_restore, True)
        kept_features = image_feature[:, 1:num_keep, :]
        restored_features[mask] = kept_features.reshape(-1, feature_dim)
        assert width * height == restored_features.shape[0], 'width * height must equal num_img'
        restored_features = restored_features.view(height, width, 24, 24, feature_dim)
        restored_features = restored_features.permute(0, 2, 1, 3, 4).contiguous()
        restored_features = restored_features.view(height, 24, width * 24, feature_dim)
        restored_features = restored_features.view(height * 24, width * 24, feature_dim)
        image_newline_expanded = self.model.image_newline.view(1, 1, feature_dim).expand(height * 24, 1, feature_dim).to(restored_features.device)
        grid_with_newline = restored_features
        mask = mask.view(height, width, 24, 24)
        mask = mask.permute(0, 2, 1, 3).contiguous()
        mask = mask.view(height * 24, width * 24)
        mask_all = mask
        image_feature_select = grid_with_newline[mask_all]
        raw_img_feature_merge = image_feature[:, -num_extra:].reshape(-1, feature_dim)
        cls_img_feature_merge = image_feature[:, 0]
        image_feature_select = torch.cat([image_feature_select, cls_img_feature_merge, raw_img_feature_merge])
        return image_feature_select

# Set up the parameters
feature_dim = 4
width = 2
height = 1
num_img = width * height  # Must satisfy the assertion
total_patches = 577  # 576 + 1 for cls token
num_keep = 10

# Create image_feature tensor with shape (num_img, total_patches, feature_dim)
image_feature = torch.randn(num_img, total_patches, feature_dim)

# Create cur_keep_idx with shape (num_img, num_keep)
# Values should be valid indices within [0, total_patches)
cur_keep_idx = torch.randint(0, total_patches, (num_img, num_keep))

# Ensure indices are sorted (as the function expects sorted indices)
cur_keep_idx, _ = cur_keep_idx.sort(dim=1)

self_obj = MinimalSelf()

ARGS = {
    'self': self_obj,
    'image_feature': image_feature,
    'cur_keep_idx': cur_keep_idx,
    'width': width,
    'height': height,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (1154, 4) — shape (1154, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
rotate_half Ran
ran; our contract draft was wrong, not the code
dvlab-research/visionzip/Qwen2_5_VL/qwen2_5vl_visionzip.py
code served (permissive licence) · get_code("e03d53ba9d4f9ae5")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# The function takes a single parameter 'x' which is a tensor
# It accesses x[..., :x.shape[-1] // 2] and x[..., x.shape[-1] // 2:]
# So x needs to be a tensor with a last dimension that can be split in half
# Let's create a small tensor with batch size 2 and feature dimension 4

x = torch.randn(2, 4)

ARGS = {
    "x": x
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
CLIPAttention_forward Not yet run
unverified (harvested, no recorded run)
dvlab-research/visionzip/visionzip/utils.py
code served (permissive licence) · get_code("994ad8442c3d6ae8")
How it ran

no invocation recorded.

apply_rotary_pos_emb_flashatt Not yet run
unverified (harvested, no recorded run)
dvlab-research/visionzip/Qwen2_5_VL/qwen2_5vl_visionzip.py
code served (permissive licence) · get_code("56baaa3cae2a8e1f")
How it ran

no invocation recorded.

apply_rotary_pos_emb_vision Not yet run
unverified (harvested, no recorded run)
dvlab-research/visionzip/Qwen2_5_VL/qwen2_5vl_visionzip.py
code served (permissive licence) · get_code("50a09accb6ca2549")
How it ran

no invocation recorded.

Also present in other papers' code

Of 11 samples lifted from this paper's linked repositories, 11 carry a function identity and 0 do not. Of the 11 distinct functions with an identity, 3 are also present in other papers whose code we hold (2 with at least one paper dated earlier, 1 the same month, 2 later) and 8 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
rotate_half 360 other papers
217 dated earlier, 10 the same month, 124 later, 6 same year, order not determinable, 3 undated · 30 only as an assembly brick
1901.02860 (2019-01)
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
2203.08913 (2022-03)
Memorizing Transformers
2302.13971 (2023-02)
LLaMA: Open and Efficient Foundation Language Models
and 357 more not named here
parse_r 4 other papers
4 dated earlier
2307.10780 (2023-07)
Learned Thresholds Token Merging and Pruning for Vision Transformers
2310.19060 (2023-10)
TESTA: Temporal-Spatial Token Aggregation for Long-form Video-Language Understanding
2403.09192 (2024-03)
PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptation
and 1 more not named here
apply_rotary_pos_emb_vision 2 other papers
2 later
2504.21403 (2025-04)
Static or Dynamic: Towards Query-Adaptive Token Selection for Video Question Answering
2509.15235 (2025-09)
ViSpec: Accelerating Vision-Language Models with Vision-Aware Speculative Decoding

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Recent advancements in vision-language models have enhanced performance by increasing the length of visual tokens, making them much longer than text tokens and significantly raising computational costs. However, we observe that the visual tokens generated by popular vision encoders, such as CLIP and SigLIP, contain significant redundancy. To address this, we introduce VisionZip, a simple yet effective method that selects a set of informative tokens for input to the language model, reducing visual token redundancy and improving efficiency while maintaining model performance. The proposed VisionZip can be widely applied to image and video understanding tasks and is well-suited for multi-turn dialogues in real-world scenarios, where previous methods tend to underperform. Experimental results show that VisionZip outperforms the previous state-of-the-art method by at least 5% performance gains across nearly all settings. Moreover, our method significantly enhances model inference speed, improving the prefilling time by 8x and enabling the LLaVA-Next 13B model to infer faster than the LLaVA-Next 7B model while achieving better results. Furthermore, we analyze the causes of this redundancy and encourage the community to focus on extracting better visual features rather than merely increasing token length. Our code is available at https://github.com/dvlab-research/VisionZip .

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2412.04467")
get_code_for_paper("2412.04467")
have("2412.04467")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2412.04467.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2412.04467.svg)](https://syntology.ai/paper/2412.04467)

Connect an agent — have() is free.