We lifted 12 functions out of this paper's own repositories and ran 4 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| gls0425/linvt | canonical | 1 of 9 |
| copy not recorded | — | 3 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| evaluate_exact_match_accuracy | Ran | this paper's copy was not recorded; identical code first harvested from brandon3964/multimodal-task-vector pointer only · get_code("e159a001bd4cbfaa") |
| evaluate_relaxed_accuracy | Ran | this paper's copy was not recorded; identical code first harvested from brandon3964/multimodal-task-vector pointer only · get_code("4ffcad52354b4806") |
| find_closest_aspect_ratio | Ran | gls0425/linvt/streamlit_demo/model_worker.py pointer only (licence: NONE) · get_code("bd77f5f8067f18e9") |
| relaxed_correctness | Ran | this paper's copy was not recorded; identical code first harvested from brandon3964/multimodal-task-vector pointer only · get_code("b3c8680ebd8739af") |
| allreduce_hook | Not yet run | gls0425/linvt/classification/ddp_hooks.py pointer only (licence: NONE) · get_code("bb1a26fe42d7075c") |
| bf16_compress_hook | Not yet run | gls0425/linvt/classification/ddp_hooks.py pointer only (licence: NONE) · get_code("21bd5be56c8f79d0") |
| build_scheduler | Not yet run | gls0425/linvt/classification/lr_scheduler.py pointer only (licence: NONE) · get_code("8c4fa54e6e3b7fb7") |
| create_logger | Not yet run | gls0425/linvt/classification/logger.py pointer only (licence: NONE) · get_code("45fcbc909d30fb3b") |
| fp16_compress_hook | Not yet run | gls0425/linvt/classification/ddp_hooks.py pointer only (licence: NONE) · get_code("26bfffb625dfe211") |
| get_config | Not yet run | gls0425/linvt/classification/config.py pointer only (licence: NONE) · get_code("e452206cf1a59c19") |
| load_image_from_base64 | Not yet run | gls0425/linvt/streamlit_demo/model_worker.py pointer only (licence: NONE) · get_code("0dde2e782959c0bd") |
| sa_flops | Not yet run | gls0425/linvt/classification/gflops.py pointer only (licence: NONE) · get_code("2dcfa48e398dddca") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Large Language Models (LLMs) have been widely used in various tasks, motivating us to develop an LLM-based assistant for videos. Instead of training from scratch, we propose a module to transform arbitrary well-trained image-based LLMs into video-LLMs (after being trained on video data). To better adapt image-LLMs for processing videos, we introduce two design principles: linear transformation to preserve the original visual-language alignment and representative information condensation from redundant video content. Guided by these principles, we propose a plug-and-play Linear Video Tokenizer(LinVT), which enables existing image-LLMs to understand videos. We benchmark LinVT with six recent visual LLMs: Aquila, Blip-3, InternVL2, Mipha, Molmo and Qwen2-VL, showcasing the high compatibility of LinVT. LinVT-based LLMs achieve state-of-the-art performance across various video benchmarks, illustrating the effectiveness of LinVT in multi-modal video understanding.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2412.05185")
get_code_for_paper("2412.05185")
have("2412.05185")
Connect an agent — have() is free.