SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2212.04501 · 2022

Learning Video Representations from Large Language Models

arXiv · PDF · Open in the Atlas

Code that ran

13 of 20 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 2 violated it and 6 have no contract verdict; 5 failures of our instrument; 7 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2212.04501.json · badge: /api/ran/2212.04501.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
facebookresearch/lavila canonical (classifier label)
classifier basis not recorded on the link
13 ran · 7 no recorded run
FunctionStatusWhere it lives
attn Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
facebookresearch/lavila/lavila/models/timesformer.py
pointer only (licence: MIT); the stored copy get_code serves, from unitaryai/VTC, is pointer only (licence: NONE) · get_code("544c604bc4f58d28")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

augment_gpt2_config Ran
ran on a synthesized input
facebookresearch/lavila/lavila/models/gpt2_gated.py
pointer only (licence: MIT) · get_code("84ff8738aacf9517")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

basic_clean Ran
ran; our contract draft was wrong, not the code
facebookresearch/lavila/lavila/models/tokenizer.py
pointer only (licence: MIT); the stored copy get_code serves, from ltpwy/MSCI, is pointer only (licence: NONE) · get_code("98f385d847636a3e")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

convert_to_normal_tensor Ran
ran on a synthesized input
facebookresearch/lavila/lavila/models/distributed_utils.py
pointer only (licence: MIT) · get_code("8e370c62b78a927f")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

default Ran
ran, violated the contract we drafted
facebookresearch/lavila/lavila/models/coca.py
pointer only (licence: MIT); the stored copy get_code serves, from ThomasMrY/VCT, is pointer only (licence: NONE) · get_code("60fff7c3c400d7ff")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

exists Ran
ran, violated the contract we drafted
facebookresearch/lavila/lavila/models/coca.py
pointer only (licence: MIT); the stored copy get_code serves, from ThomasMrY/VCT, is pointer only (licence: NONE) · get_code("aa5486a3650902d8")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

gather_features Ran
ran on a synthesized input
facebookresearch/lavila/lavila/models/loss.py
pointer only (licence: MIT) · get_code("19519e7e08dd0087")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

gather_from_all Ran
ran on a synthesized input
facebookresearch/lavila/lavila/models/distributed_utils.py
pointer only (licence: MIT) · get_code("b57f5d4b094326f8")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

get_metric_names Ran
ran on a synthesized input
facebookresearch/lavila/lavila/models/models.py
pointer only (licence: MIT) · get_code("051362c936e3619d")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

get_pairs Ran
ran; our contract draft was wrong, not the code
facebookresearch/lavila/lavila/models/tokenizer.py
pointer only (licence: MIT) · get_code("d919ae32e5e4e616")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

inflate_positional_embeds Ran
ran on a synthesized input
facebookresearch/lavila/lavila/models/utils.py
pointer only (licence: MIT) · get_code("d8e9e992e4668cbb")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

sim_matrix Ran
ran; our contract draft was wrong, not the code
facebookresearch/lavila/lavila/models/loss.py
pointer only (licence: MIT) · get_code("da8e97ddc50728ed")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

whitespace_clean Ran
ran; our contract draft was wrong, not the code
facebookresearch/lavila/lavila/models/tokenizer.py
pointer only (licence: MIT); the stored copy get_code serves, from ltpwy/MSCI, is pointer only (licence: NONE) · get_code("9542161e9640b858")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

build_model Not yet run
unverified (harvested, no recorded run)
facebookresearch/lavila/lavila/models/openai_model.py
pointer only (licence: MIT) · get_code("7144c613fbeb2f92")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

convert_to_distributed_tensor Not yet run
unverified (harvested, no recorded run)
facebookresearch/lavila/lavila/models/distributed_utils.py
pointer only (licence: MIT) · get_code("bcfb2b11cc600666")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

get_model Not yet run
unverified (harvested, no recorded run)
facebookresearch/lavila/lavila/utils/distributed.py
pointer only (licence: MIT); the stored copy get_code serves, from camma-public/peskavlp, is pointer only (licence: NONE) · get_code("2ec9ac675733a9ac")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

load Not yet run
unverified (harvested, no recorded run)
facebookresearch/lavila/lavila/models/openai_clip.py
pointer only (licence: MIT) · get_code("97742b8486aa64f2")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

load_tf_weights_in_gpt2 Not yet run
unverified (harvested, no recorded run)
facebookresearch/lavila/lavila/models/gpt2_gated.py
pointer only (licence: MIT) · get_code("bc9fb74e483fb901")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

rgetattr Not yet run
unverified (harvested, no recorded run)
facebookresearch/lavila/lavila/models/utils.py
pointer only (licence: MIT) · get_code("44925cf3831bfe79")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

rsetattr Not yet run
unverified (harvested, no recorded run)
facebookresearch/lavila/lavila/models/utils.py
pointer only (licence: MIT); the stored copy get_code serves, from DBO-DKFZ/uncertainty-benchmark, is pointer only (licence: NONE) · get_code("f9de4cdd8a624801")
How it ran

The body of this sample is withheld: MIT, as GitHub detects it on this paper's repository, facebookresearch/lavila, is in the permissive set, but a licence decision keeps the body a pointer: the reason for that decision is not recorded with it. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

Also present in other papers' code

Of 20 samples lifted from this paper's linked repositories, 20 carry a function identity and 0 do not. Of the 20 distinct functions with an identity, 13 are also present in other papers whose code we hold (12 with at least one paper dated earlier, 5 the same month, 13 later) and 7 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
exists 486 other papers
130 dated earlier, 6 the same month, 347 later, 1 same year, order not determinable, 2 undated · 78 only as an assembly brick
1603.08155 (2016-03)
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
1611.07004 (2016-11)
Image-to-Image Translation with Conditional Adversarial Networks
1804.04637 (2018-04)
EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models
and 483 more not named here
get_pairs 258 other papers
35 dated earlier, 5 the same month, 217 later, 1 undated · 4 only as an assembly brick
1802.06474 (2018-02)
A Closed-form Solution to Photorealistic Image Stylization
1803.01128 (2018-03)
Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples
1809.00125 (2018-09)
Simple Fusion: Return of the Language Model
and 255 more not named here
whitespace_clean 231 other papers
21 dated earlier, 5 the same month, 204 later, 1 undated · 4 only as an assembly brick
1901.07441 (2019-01)
PadChest: A large chest x-ray image dataset with multi-label annotated reports
2107.06383 (2021-07)
How Much Can CLIP Benefit Vision-and-Language Tasks?
2108.10904 (2021-08)
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision
and 228 more not named here
basic_clean 218 other papers
21 dated earlier, 5 the same month, 191 later, 1 undated · 4 only as an assembly brick
1901.07441 (2019-01)
PadChest: A large chest x-ray image dataset with multi-label annotated reports
2107.06383 (2021-07)
How Much Can CLIP Benefit Vision-and-Language Tasks?
2108.10904 (2021-08)
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision
and 215 more not named here
default 163 other papers
64 dated earlier, 3 the same month, 95 later, 1 undated · 27 only as an assembly brick
1603.08155 (2016-03)
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
1810.00825 (2018-10)
Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks
1907.01470 (2019-07)
Augmenting Self-attention with Persistent Memory
and 160 more not named here
build_model 53 other papers
3 dated earlier, 50 later
2107.06383 (2021-07)
How Much Can CLIP Benefit Vision-and-Language Tasks?
2210.01115 (2022-10)
LASP: Text-to-Text Optimization for Language-Aware Soft Prompting of Vision & Language Models
2211.09552 (2022-11)
UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormer
and 50 more not named here
load 23 other papers
2 dated earlier, 20 later, 1 undated
2208.05318 (2022-08)
Generative Action Description Prompts for Skeleton-based Action Recognition
2210.01115 (2022-10)
LASP: Text-to-Text Optimization for Language-Aware Soft Prompting of Vision & Language Models
2302.02334 (2023-02)
Revisiting Discriminative vs. Generative Classifiers: Theory and Implications
and 20 more not named here
rgetattr 13 other papers
3 dated earlier, 10 later · 1 only as an assembly brick
2203.04036 (2022-03)
StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN
2209.02952 (2022-09)
BiFuse++: Self-supervised and Efficient Bi-projection Fusion for 360 Depth Estimation
2211.05778 (2022-11)
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions
and 10 more not named here
load_tf_weights_in_gpt2 11 other papers
3 dated earlier, 8 later
2206.11871 (2022-06)
Offline RL for Natural Language Generation with Implicit Language Q Learning
2211.09817 (2022-11)
On the Effect of Pre-training for Transformer in Different Modality on Offline Reinforcement Learning
2211.11694 (2022-11)
Exploring Discrete Diffusion Models for Image Captioning
and 8 more not named here
attn 8 other papers
4 dated earlier, 4 later · 2 only as an assembly brick
2102.05095 (2021-02)
Is Space-Time Attention All You Need for Video Understanding?
2104.00650 (2021-04)
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval
2201.04850 (2022-01)
Bridging Video-text Retrieval with Multiple Choice Questions
and 5 more not named here
rsetattr 8 other papers
1 dated earlier, 7 later
2203.04036 (2022-03)
StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN
2305.13417 (2023-05)
VISIT: Visualizing and Interpreting the Semantic Information Flow of Transformers
2311.12871 (2023-11)
An Embodied Generalist Agent in 3D World
and 5 more not named here
sim_matrix 8 other papers
2 dated earlier, 5 later, 1 same year, order not determinable
2104.00650 (2021-04)
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval
2022.findings-emnlp.24 (2022 (year only))
AssistSR: Task-oriented Video Segment Retrieval for Personal AI Assistant
2206.01670 (2022-06)
Egocentric Video-Language Pretraining
and 5 more not named here
get_model 4 other papers
4 later
Yang_Attentive_Mask_CLIP_ICCV_2023_paper (2023 (year only))
Attentive Mask CLIP
2305.20088 (2023-05)
Improving CLIP Training with Language Rewrites
2402.01832 (2024-02)
SynthCLIP: Are We Ready for a Fully Synthetic CLIP Training?
and 1 more not named here

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We introduce LaViLa, a new approach to learning video-language representations by leveraging Large Language Models (LLMs). We repurpose pre-trained LLMs to be conditioned on visual input, and finetune them to create automatic video narrators. Our auto-generated narrations offer a number of advantages, including dense coverage of long videos, better temporal synchronization of the visual information and text, and much higher diversity of text. The video-text embedding learned contrastively with these additional auto-generated narrations outperforms the previous state-of-the-art on multiple first-person and third-person video tasks, both in zero-shot and finetuned setups. Most notably, LaViLa obtains an absolute gain of 10.1% on EGTEA classification and 5.9% Epic-Kitchens-100 multi-instance retrieval benchmarks. Furthermore, LaViLa trained with only half the narrations from the Ego4D dataset outperforms baseline models trained on the full set, and shows positive scaling behavior on increasing pre-training data and model size.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2212.04501")
get_code_for_paper("2212.04501")
have("2212.04501")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2212.04501.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2212.04501.svg)](https://syntology.ai/paper/2212.04501)

Connect an agent — have() is free.