Jan Kautz, U Berkeley, Song Han, Trevor Darrell, Han Cai, Baifeng Shi, Marco Pavone, Sifei Liu, Yao Lu, Boyi Li, Hongxu Yin, Pavlo Molchanov, and 1 more
We lifted 14 functions out of this paper's own repositories and ran 11 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| efficient-large-model/vila | canonical | 10 of 12 |
| nvlabs/vila | — | 1 of 2 |
| Function | Status | Where it lives |
|---|---|---|
| check_dot_in_model_path | Ran | efficient-large-model/vila/llava/remote_code/modeling_vila.py code served (permissive licence) · get_code("8dd6b65f72c07727") |
| repeat_kv | Ran | efficient-large-model/vila/llava/eval/vision_niah_vila/zigzag_ring_attn/modeling_qwen2.py code served (permissive licence) · get_code("30d7eec482ebf6b1") |
| VisionTower | Ran | nvlabs/vila/llava/model/multimodal_encoder/ps3_encoder.py code served (permissive licence) · get_code("232455d9c7b1d476") |
| apply_rotary_pos_emb | Ran | efficient-large-model/vila/llava/eval/vision_niah_vila/zigzag_ring_attn/modeling_qwen2.py code served (permissive licence) · get_code("5d81bffdb6022427") |
| get_vila_version | Ran | efficient-large-model/vila/llava/remote_code/modeling_vila.py code served (permissive licence) · get_code("554c742ab091b991") |
| load_haystack | Ran | efficient-large-model/vila/llava/eval/vision_niah_vila/eval_vision_niah.py code served (permissive licence) · get_code("70759df34aae94c5") |
| load_image | Ran | efficient-large-model/vila/server.py code served (permissive licence) · get_code("1c69e761fdf05a26") |
| load_video | Ran | efficient-large-model/vila/server.py code served (permissive licence) · get_code("fca9a6d2afa23288") |
| rotate_half | Ran | efficient-large-model/vila/llava/eval/vision_niah_vila/zigzag_ring_attn/modeling_qwen2.py code served (permissive licence) · get_code("b99eea6376d1e212") |
| safe_tokenize | Ran | efficient-large-model/vila/llava/eval/vision_niah_vila/eval_vision_niah.py code served (permissive licence) · get_code("300bb78b66d35115") |
| with_opencv | Ran | efficient-large-model/vila/data_prepare/panda_split.py code served (permissive licence) · get_code("7af38f0a2846676a") |
| PS3VisionTower | Not yet run | nvlabs/vila/llava/model/multimodal_encoder/ps3_encoder.py code served (permissive licence) · get_code("ef6066f5b1710c5d") |
| build_mm_projector | Not yet run | efficient-large-model/vila/llava/remote_code/modeling_vila.py code served (permissive licence) · get_code("aa7f4da9de0dc7cc") |
| get_literal_values | Not yet run | efficient-large-model/vila/server.py code served (permissive licence) · get_code("f8e6d235cb12b138") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Figure 1. Left: Regular vision models such as SigLIP [44] processes images at a low resolution (e.g., 378 × 378 pixels), which is not enough for many daily tasks such as spotting the stop sign while driving. In contrast, PS3 is able to both encode low-res features and efficiently process high-res information of 4K-resolution images via top-down patch selection, i.e., selectively processing relevant patches based on any text prompt. Top Right: SigLIP is pre-trained by contrasting global vision features and global captions, which is costly for high-resolution images. PS3 is pre-trained with additional contrast between local high-res features with local captions, enabling pre-training at 4K resolution with 79× less cost than SigLIP. Bottom Right: VILA-HD uses PS3 to selectively process high-res regions based on the user prompt, outperforming state-of-the-art MLLMs such as Qwen2-VL [38] on the proposed 4KPro benchmark while achieving 2.96× speedup.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2503.19903")
get_code_for_paper("2503.19903")
have("2503.19903")
Connect an agent — have() is free.