Bo Pang, Jack Hessel, Radu Soricut, Zhenhai Zhu
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Pretraining from unlabelled web videos has quickly become the de-facto means of achieving high performance on many video understanding tasks. Features are learned via prediction of grounded relationships between visual content and automatic speech recognition (ASR) tokens. However, prior pretraining work has been limited to only instructional videos; a priori, we expect this domain to be relatively "easy:" speakers in instructional videos will often reference the literal objects/actions being depicted. We ask: can similar models be trained on more diverse video corpora? And, if so, what types of videos are "grounded" and what types are not? We fit a representative pretraining model to the diverse YouTube8M dataset, and study its success and failure cases. We find that visualtextual grounding is indeed possible across previously unexplored video categories, and that pretraining on a more diverse set results in representations that generalize to both noninstructional and instructional domains.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2004.14338")
get_code_for_paper("2004.14338")
have("2004.14338")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2004.14338.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2004.14338)
Connect an agent — have() is free.