SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2402.19082 · CVPR · 2024

VideoMAC: Video Masked Autoencoders Meet ConvNets

Yazhou Yao, Tao Chen, Huafeng Liu, Zeren Sun, Gensheng Pei, Xiruo Jiang

arXiv · PDF · Open in the Atlas

Code that ran

We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Recently, the advancement of self-supervised learning techniques, like masked autoencoders (MAE), has greatly influenced visual representation learning for images and videos. Nevertheless, it is worth noting that the predominant approaches in existing masked image / video modeling rely excessively on resource-intensive vision transformers (ViTs) as the feature encoder. In this paper, we propose a new approach termed as VideoMAC, which combines video masked autoencoders with resource-friendly Con-vNets. Specifically, VideoMAC employs symmetric masking on randomly sampled pairs of video frames. To prevent the issue of mask pattern dissipation, we utilize ConvNets which are implemented with sparse convolutional operators as encoders. Simultaneously, we present a simple yet effective masked video modeling (MVM) approach, a dual encoder architecture comprising an online encoder and an exponential moving average target encoder, aimed to facilitate inter-frame reconstruction consistency in videos. Additionally, we demonstrate that VideoMAC, empowering classical (ResNet) / modern (ConvNeXt) convolutional encoders to harness the benefits of MVM, outperforms ViT-based approaches on downstream tasks, including video object segmentation (+5.2% / 6.4% J &F), body part propagation (+6.3% / 3.1% mIoU), and human pose tracking (+10.2% / 11.1%

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2402.19082")
get_code_for_paper("2402.19082")
have("2402.19082")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2402.19082.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2402.19082.svg)](https://syntology.ai/paper/2402.19082)

Connect an agent — have() is free.