We lifted 13 functions out of this paper's own repositories and ran 8 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| opengvlab/video-mamba-suite | canonical | 8 of 13 |
| Function | Status | Where it lives |
|---|---|---|
| crop_clip | Ran | opengvlab/video-mamba-suite/video-mamba-suite/action-recognition/functional.py code served (permissive licence) · get_code("bc39b8ed5fefa7eb") |
| exponential_descrease | Ran | opengvlab/video-mamba-suite/video-mamba-suite/temporal-action-segmentation/model.py code served (permissive licence) · get_code("8467c56ed6949a06") |
| get_loss_scale_for_deepspeed | Ran | opengvlab/video-mamba-suite/video-mamba-suite/action-recognition/engines/engine_for_finetuning.py code served (permissive licence) · get_code("6aef977b44c0feaf") |
| get_resize_sizes | Ran | opengvlab/video-mamba-suite/video-mamba-suite/action-recognition/functional.py code served (permissive licence) · get_code("2b53e457d5632b4e") |
| get_sinusoid_encoding_table | Ran | opengvlab/video-mamba-suite/video-mamba-suite/action-recognition/models/modeling_finetune.py code served (permissive licence) · get_code("f9c6fc8c1ddfbfac") |
| resize_clip | Ran | opengvlab/video-mamba-suite/video-mamba-suite/action-recognition/functional.py code served (permissive licence) · get_code("981936c73d3cdd12") |
| spatial_sampling | Ran | opengvlab/video-mamba-suite/video-mamba-suite/action-recognition/datasets/kinetics.py code served (permissive licence) · get_code("f188ca6f84532f7f") |
| train_class_batch | Ran | opengvlab/video-mamba-suite/video-mamba-suite/action-recognition/engines/engine_for_finetuning_regression.py code served (permissive licence) · get_code("fb98703650f02bf6") |
| clip_b16 | Not yet run | opengvlab/video-mamba-suite/video-mamba-suite/action-recognition/models/clip.py code served (permissive licence) · get_code("e72f71197ef8b97e") |
| clip_l14 | Not yet run | opengvlab/video-mamba-suite/video-mamba-suite/action-recognition/models/clip.py code served (permissive licence) · get_code("055a054d676fb93c") |
| inflate_weight | Not yet run | opengvlab/video-mamba-suite/video-mamba-suite/action-recognition/models/clip.py code served (permissive licence) · get_code("bdec7f112532de68") |
| tensor_normalize | Not yet run | opengvlab/video-mamba-suite/video-mamba-suite/action-recognition/datasets/kinetics.py code served (permissive licence) · get_code("14b3b13b2e7ee844") |
| train_class_batch | Not yet run | opengvlab/video-mamba-suite/video-mamba-suite/action-recognition/engines/engine_for_finetuning.py code served (permissive licence) · get_code("c7976ea27edc377a") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Understanding videos is one of the fundamental directions in computer vision research, with extensive efforts dedicated to exploring various architectures such as RNN, 3D CNN, and Transformers. The newly proposed architecture of state space model, e.g., Mamba, shows promising traits to extend its success in long sequence modeling to video modeling. To assess whether Mamba can be a viable alternative to Transformers in the video understanding domain, in this work, we conduct a comprehensive set of studies, probing different roles Mamba can play in modeling videos, while investigating diverse tasks where Mamba could exhibit superiority. We categorize Mamba into four roles for modeling videos, deriving a Video Mamba Suite composed of 14 models/modules, and evaluating them on 12 video understanding tasks. Our extensive experiments reveal the strong potential of Mamba on both video-only and video-language tasks while showing promising efficiency-performance trade-offs. We hope this work could provide valuable data points and insights for future research on video understanding. Code is public: https://github.com/OpenGVLab/video-mamba-suite.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2403.09626")
get_code_for_paper("2403.09626")
have("2403.09626")
Connect an agent — have() is free.