Dongho Lee, Jongseo Lee, Jinwoo Choi
We lifted 17 functions out of this paper's own repositories and ran 9 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| KHU-VLL/CAST | canonical | 5 of 9 |
| khu-vll/cast | — | 4 of 8 |
| Function | Status | Where it lives |
|---|---|---|
| Attention | Ran | khu-vll/cast/models/bidir_modeling_crossattn.py pointer only (licence: NOASSERTION) · get_code("b181227b8cc95f3d") |
| CrossAttentionT2S | Ran | khu-vll/cast/models/bidir_modeling_crossattn.py pointer only (licence: NOASSERTION) · get_code("ef0019b3743b17db") |
| LayerNorm | Ran | khu-vll/cast/models/bidir_modeling_crossattn.py pointer only (licence: NOASSERTION) · get_code("353426ba46a4932c") |
| composition_train_class_batch | Ran | KHU-VLL/CAST/engine_for_compomodel.py pointer only (licence: NOASSERTION) · get_code("457807b46c3549fe") |
| crop_clip | Ran | KHU-VLL/CAST/util_tools/functional.py pointer only (licence: NONE) · get_code("bc39b8ed5fefa7eb") |
| cross_train_class_batch | Ran | KHU-VLL/CAST/engine_for_onemodel.py pointer only (licence: NOASSERTION) · get_code("2604141544b5eec0") |
| get_resize_sizes | Ran | KHU-VLL/CAST/util_tools/functional.py pointer only (licence: NONE) · get_code("2b53e457d5632b4e") |
| get_sinusoid_encoding_table | Ran | khu-vll/cast/models/bidir_modeling_crossattn.py pointer only (licence: NOASSERTION) · get_code("9b2250a3d13ed688") |
| resize_clip | Ran | KHU-VLL/CAST/util_tools/functional.py pointer only (licence: NONE) · get_code("981936c73d3cdd12") |
| Adapter | Not yet run | khu-vll/cast/models/bidir_modeling_crossattn.py pointer only (licence: NOASSERTION) · get_code("857af2a7de44faaa") |
| Block | Not yet run | khu-vll/cast/models/bidir_modeling_crossattn.py pointer only (licence: NOASSERTION) · get_code("1dd28bf10048de7d") |
| CrossAttentionS2T | Not yet run | khu-vll/cast/models/bidir_modeling_crossattn.py pointer only (licence: NOASSERTION) · get_code("445bb3da9e1748a4") |
| STCrossTransformer | Not yet run | khu-vll/cast/models/bidir_modeling_crossattn.py pointer only (licence: NOASSERTION) · get_code("d26fe19969fbc2c2") |
| build_dataset | Not yet run | KHU-VLL/CAST/dataset/datasets.py pointer only (licence: NOASSERTION) · get_code("bc71b534fe126be4") |
| clip_vit_base_patch16_224 | Not yet run | KHU-VLL/CAST/models/clip_modelling_finetune.py pointer only (licence: NOASSERTION) · get_code("76274d93634ef7d9") |
| get_loss_scale_for_deepspeed | Not yet run | KHU-VLL/CAST/engine_for_compomodel.py pointer only (licence: NONE) · get_code("29416043c7035e7c") |
| tensor_normalize | Not yet run | KHU-VLL/CAST/dataset/kinetics.py pointer only (licence: NONE) · get_code("14b3b13b2e7ee844") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Recognizing human actions in videos requires spatial and temporal understanding. Most existing action recognition models lack a balanced spatio-temporal understanding of videos. In this work, we propose a novel two-stream architecture, called Cross-Attention in Space and Time (CAST), that achieves a balanced spatio-temporal understanding of videos using only RGB input. Our proposed bottleneck cross-attention mechanism enables the spatial and temporal expert models to exchange information and make synergistic predictions, leading to improved performance. We validate the proposed method with extensive experiments on public benchmarks with different characteristics: EPIC-KITCHENS-100, Something-Something-V2, and Kinetics-400. Our method consistently shows favorable performance across these datasets, while the performance of existing methods fluctuates depending on the dataset characteristics. The code is available at https://github.com/KHU-VLL/CAST.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2311.18825")
get_code_for_paper("2311.18825")
have("2311.18825")
Connect an agent — have() is free.