5 of 9 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 2 violated it and 1 has no contract verdict; 2 failures of our instrument; 4 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.
"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2405.21075.json · badge: /api/ran/2405.21075.svg
Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.
| Repository | Role | Ran |
|---|---|---|
| PhysGame/PhysGame | community (classifier label) classifier basis not recorded on the link |
5 ran · 4 no recorded run |
| Function | Status | Where it lives |
|---|---|---|
| disabled_train | Ran ran, violated the contract we drafted |
PhysGame/PhysGame/physvlm/models/blip2.py code served (permissive licence) · get_code("4cb732f513d69dfd") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned SimpleModel. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
|
| drop_path | Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
PhysGame/PhysGame/physvlm/models/clip_btadapter.py code served (permissive licence) · get_code("9a83c0663b6afcd4") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 4, dtype family float, shape (2, 4, 3, 3) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
|
| get_index | Ran ran on a synthesized input |
PhysGame/PhysGame/demo_gradio.py code served (permissive licence) · get_code("fef447001a2f36dc") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family int, shape (32,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
|
| is_transformers_greater_than | Ran ran, violated the contract we drafted |
PhysGame/PhysGame/physvlm/import_utils.py code served (permissive licence) · get_code("f4334934ee795269") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
|
| tile | Ran ran; our contract draft was wrong, not the code |
PhysGame/PhysGame/physvlm/models/base_model.py code served (permissive licence) · get_code("ded9fb4cd3944865") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 6) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
|
| all_gather_with_grad | Not yet run unverified (harvested, no recorded run) |
PhysGame/PhysGame/physvlm/models/base_model.py code served (permissive licence) · get_code("0ec9fc2025c16f65") How it ranno invocation recorded. |
| get_peft_state_maybe_zero_3 | Not yet run unverified (harvested, no recorded run) |
PhysGame/PhysGame/train_dpo.py code served (permissive licence) · get_code("fa1225dfac92bc0d") How it ranno invocation recorded. |
| get_peft_state_non_lora_maybe_zero_3 | Not yet run unverified (harvested, no recorded run) |
PhysGame/PhysGame/train_dpo.py code served (permissive licence) · get_code("1c53657305b66e9f") How it ranno invocation recorded. |
| maybe_zero_3 | Not yet run unverified (harvested, no recorded run) |
PhysGame/PhysGame/train_dpo.py code served (permissive licence) · get_code("616ffbdc154ed2d8") How it ranno invocation recorded. |
Of 9 samples lifted from this paper's linked repositories, 9 carry a function identity and 0 do not. Of the 9 distinct functions with an identity, 8 are also present in other papers whose code we hold (6 with at least one paper dated earlier, 6 the same month, 8 later) and 1 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).
Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.
| Function | Also present in | Earliest-dated | Others named |
|---|---|---|---|
| disabled_train | 173 other papers 104 dated earlier, 12 the same month, 49 later, 8 same year, order not determinable · 5 only as an assembly brick |
2012.09841 (2020-12) Taming Transformers for High-Resolution Image Synthesis |
2102.05379 (2021-02) Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions 2104.07652 (2021-04) Geometry-Free View Synthesis: Transformers and no 3D Priors and 170 more not named here |
| all_gather_with_grad | 91 other papers 59 dated earlier, 4 the same month, 24 later, 4 same year, order not determinable · 2 only as an assembly brick |
2206.02967 (2022-06) Masked Unsupervised Self-training for Label-free Image Classification |
2210.08773 (2022-10) Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training 2302.06605 (2023-02) UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling and 88 more not named here |
| maybe_zero_3 | 37 other papers 10 dated earlier, 2 the same month, 25 later |
2310.01779 (2023-10) HallE-Control: Controlling Object Hallucination in Large Multimodal Models |
2311.16839 (2023-11) Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization 2312.02949 (2023-12) LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models and 34 more not named here |
| get_peft_state_maybe_zero_3 | 23 other papers 4 dated earlier, 2 the same month, 17 later |
2310.01779 (2023-10) HallE-Control: Controlling Object Hallucination in Large Multimodal Models |
2401.02330 (2024-01) LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model 2402.12501 (2024-02) Your Vision-Language Model Itself Is a Strong Filter: Towards High-Quality Instruction Tuning with Data Selection and 20 more not named here |
| get_peft_state_non_lora_maybe_zero_3 | 23 other papers 5 dated earlier, 1 the same month, 17 later |
2310.01779 (2023-10) HallE-Control: Controlling Object Hallucination in Large Multimodal Models |
2312.02949 (2023-12) LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models 2401.02330 (2024-01) LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model and 20 more not named here |
| is_transformers_greater_than | 4 other papers 1 dated earlier, 1 the same month, 2 later |
2404.14233 (2024-04) Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback |
2405.18654 (2024-05) Mitigating Object Hallucination in MLLMs via Data-augmented Phrase-level Alignment 2410.06101 (2024-10) Coevolving with the Other You: Fine-Tuning LLM with Sequential Cooperative Multi-Agent Reinforcement Learning and 1 more not named here |
| drop_path | 1 other paper 1 later |
2411.02327 (2024-11) PPLLaVA: Varied Video Sequence Understanding With Prompt Guidance |
|
| tile | 1 other paper 1 later |
2411.02327 (2024-11) PPLLaVA: Varied Video Sequence Understanding With Prompt Guidance |
Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
In the quest for artificial general intelligence, Multi-modal Large Language Models (MLLMs) have emerged as a focal point in recent advancements. However, the predominant focus remains on developing their capabilities in static image understanding. The potential of MLLMs in processing sequential visual data is still insufficiently explored, highlighting the absence of a comprehensive, high-quality assessment of their performance. In this paper, we introduce Video-MME, the first-ever full-spectrum, Multi-Modal Evaluation benchmark of MLLMs in Video analysis. Our work distinguishes from existing benchmarks through four key features: 1) Diversity in video types, spanning 6 primary visual domains with 30 subfields to ensure broad scenario generalizability; 2) Duration in temporal dimension, encompassing both short-, medium-, and long-term videos, ranging from 11 seconds to 1 hour, for robust contextual dynamics; 3) Breadth in data modalities, integrating multi-modal inputs besides video frames, including subtitles and audios, to unveil the all-round capabilities of MLLMs; 4) Quality in annotations, utilizing rigorous manual labeling by expert annotators to facilitate precise and reliable model assessment. 900 videos with a total of 254 hours are manually selected and annotated by repeatedly viewing all the video content, resulting in 2,700 question-answer pairs. With Video-MME, we extensively evaluate various state-of-the-art MLLMs, including GPT-4 series and Gemini 1.5 Pro, as well as open-source image models like InternVL-Chat-V1.5 and video models like LLaVA-NeXT-Video. Our experiments reveal that Gemini 1.5 Pro is the best-performing commercial model, significantly outperforming the open-source models. Our dataset along with these findings underscores the need for further improvements in handling longer sequences and multi-modal data. Project Page: https://video-mme.github.io
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2405.21075")
get_code_for_paper("2405.21075")
have("2405.21075")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2405.21075.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2405.21075)
Connect an agent — have() is free.