SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2004.14338 · EMNLP · 2020

Beyond Instructional Videos: Probing for More Diverse Visual-Textual Grounding on YouTube

Bo Pang, Jack Hessel, Radu Soricut, Zhenhai Zhu

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Pretraining from unlabelled web videos has quickly become the de-facto means of achieving high performance on many video understanding tasks. Features are learned via prediction of grounded relationships between visual content and automatic speech recognition (ASR) tokens. However, prior pretraining work has been limited to only instructional videos; a priori, we expect this domain to be relatively "easy:" speakers in instructional videos will often reference the literal objects/actions being depicted. We ask: can similar models be trained on more diverse video corpora? And, if so, what types of videos are "grounded" and what types are not? We fit a representative pretraining model to the diverse YouTube8M dataset, and study its success and failure cases. We find that visualtextual grounding is indeed possible across previously unexplored video categories, and that pretraining on a more diverse set results in representations that generalize to both noninstructional and instructional domains.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2004.14338")
get_code_for_paper("2004.14338")
have("2004.14338")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2004.14338.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2004.14338.svg)](https://syntology.ai/paper/2004.14338)

Connect an agent — have() is free.