SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2104.11746 · ICCV · 2021

VidTr: Video Transformer Without Convolutions

Hao Chen, Xinyu Li, Yi Zhu, Bing Shuai, Joseph Tighe, Yanyi Zhang, Chunhui Liu, Biagio Brattoli, Ivan Marsic

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. We have no repository linked to it either. That is a statement about our graph, not about the paper.

Abstract

We introduce Video Transformer (VidTr) with separableattention for video classification. Comparing with commonly used 3D networks, VidTr is able to aggregate spatiotemporal information via stacked attentions and provide better performance with higher efficiency. We first introduce the vanilla video transformer and show that transformer module is able to perform spatio-temporal modeling from raw pixels, but with heavy memory usage. We then present VidTr which reduces the memory cost by 3.3× while keeping the same performance. To further optimize the model, we propose the standard deviation based topK pooling for attention (pool topK std ), which reduces the computation by dropping non-informative features along temporal dimension. VidTr achieves state-of-the-art performance on five commonly used datasets with lower computational requirement, showing both the efficiency and effectiveness of our design. Finally, error analysis and visualization show that VidTr is especially good at predicting actions that require long-term temporal reasoning.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2104.11746")
get_code_for_paper("2104.11746")
have("2104.11746")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2104.11746.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2104.11746.svg)](https://syntology.ai/paper/2104.11746)

Connect an agent — have() is free.