We lifted 3 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| jimmy646/violin | canonical | 3 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| clean_str | Ran | jimmy646/violin/violin_dataset.py code served (permissive licence) · get_code("cd95678702d04756") |
| pad_collate | Ran | jimmy646/violin/violin_dataset.py code served (permissive licence) · get_code("c35ac9d938aa83bc") |
| preprocess_batch | Ran | jimmy646/violin/violin_dataset.py code served (permissive licence) · get_code("ef63a3f42a6e6f4f") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We introduce a new task, Video-and-Language Inference, for joint multimodal understanding of video and text. Given a video clip with aligned subtitles as premise, paired with a natural language hypothesis based on the video content, a model needs to infer whether the hypothesis is entailed or contradicted by the given video clip. A new large-scale dataset, named Violin (VIdeO-and-Language INference), is introduced for this task, which consists of 95,322 video-hypothesis pairs from 15,887 video clips, spanning over 582 hours of video. These video clips contain rich content with diverse temporal dynamics, event shifts, and people interactions, collected from two sources: (i) popular TV shows, and (ii) movie clips from YouTube channels. In order to address our new multimodal inference task, a model is required to possess sophisticated reasoning skills, from surface-level grounding (e.g., identifying objects and characters in the video) to in-depth commonsense reasoning (e.g., inferring causal relations of events in the video). We present a detailed analysis of the dataset and an extensive evaluation over many strong baselines, providing valuable insights on the challenges of this new task.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2003.11618")
get_code_for_paper("2003.11618")
have("2003.11618")
Connect an agent — have() is free.