We lifted 13 functions out of this paper's own repositories and ran 11 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| ilpoviertola/V-AURA | canonical | 11 of 13 |
| Function | Status | Where it lives |
|---|---|---|
| crop_or_pad_tensor | Ran | ilpoviertola/V-AURA/utils/data_utils.py code served (permissive licence) · get_code("78574c290e72fec5") |
| drop_path | Ran | ilpoviertola/V-AURA/utils/drop_path.py code served (permissive licence) · get_code("87577b3ff9d32712") |
| find_multiple | Ran | ilpoviertola/V-AURA/models/modules/sampler/llama.py code served (permissive licence) · get_code("f6ff7671338c9c92") |
| get_latest_file | Ran | ilpoviertola/V-AURA/utils/utils.py code served (permissive licence) · get_code("d8d3543672647554") |
| get_obj_from_str | Ran | ilpoviertola/V-AURA/utils/utils.py code served (permissive licence) · get_code("221b2d116fdf1032") |
| init_log_directory | Ran | ilpoviertola/V-AURA/utils/train_utils.py code served (permissive licence) · get_code("9504a5d5fd37c7c4") |
| is_master | Ran | ilpoviertola/V-AURA/utils/train_utils.py code served (permissive licence) · get_code("5f3e6dae89c753a1") |
| nullify_condition | Ran | ilpoviertola/V-AURA/models/modules/misc/dropout_modules.py code served (permissive licence) · get_code("4d5609b08d1af0a9") |
| precompute_freqs_cis | Ran | ilpoviertola/V-AURA/models/modules/sampler/llama.py code served (permissive licence) · get_code("2c69b343e0f9b255") |
| precompute_freqs_cis_2d | Ran | ilpoviertola/V-AURA/models/modules/sampler/llama.py code served (permissive licence) · get_code("7f53b2cdcd16fad9") |
| read_video_to_frames_and_audio_streams | Ran | ilpoviertola/V-AURA/utils/data_utils.py code served (permissive licence) · get_code("f994ce3168a6e9da") |
| instantiate_from_config | Not yet run | ilpoviertola/V-AURA/utils/utils.py code served (permissive licence) · get_code("692c94009ed723bc") |
| loadvideo_decord | Not yet run | ilpoviertola/V-AURA/utils/data_utils.py code served (permissive licence) · get_code("0e135ebdd6667856") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We introduce V-AURA, the first autoregressive model to achieve high temporal alignment and relevance in video-to-audio generation. V-AURA uses a high-framerate visual feature extractor and a cross-modal audio-visual feature fusion strategy to capture fine-grained visual motion events and ensure precise temporal alignment. Additionally, we propose VisualSound, a benchmark dataset with high audio-visual relevance. VisualSound is based on VGGSound, a video dataset consisting of in-the-wild samples extracted from YouTube. During the curation, we remove samples where auditory events are not aligned with the visual ones. V-AURA outperforms current state-of-the-art models in temporal alignment and semantic relevance while maintaining comparable audio quality. Code, samples, VisualSound and models are available at https://v-aura.notion.site
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2409.13689")
get_code_for_paper("2409.13689")
have("2409.13689")
Connect an agent — have() is free.