We lifted 16 functions out of this paper's own repositories and ran 11 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| exponentialml/text-to-video-finetuning | canonical | 11 of 16 |
| Function | Status | Where it lives |
|---|---|---|
| convert_text_enc_state_dict | Ran | exponentialml/text-to-video-finetuning/utils/convert_diffusers_to_original_ms_text_to_video.py code served (permissive licence) · get_code("a59539766c1c5a5a") |
| convert_text_enc_state_dict_v20 | Ran | exponentialml/text-to-video-finetuning/utils/convert_diffusers_to_original_ms_text_to_video.py code served (permissive licence) · get_code("8d4b0c9f8b9a884e") |
| download_progress | Ran | exponentialml/text-to-video-finetuning/utils/lama.py code served (permissive licence) · get_code("bfdb56f5b6ab8034") |
| filter_dict | Ran | exponentialml/text-to-video-finetuning/utils/lora_handler.py code served (permissive licence) · get_code("9191602362628ded") |
| get_bucket_sizes | Ran | exponentialml/text-to-video-finetuning/utils/bucketing.py code served (permissive licence) · get_code("8ff113de21fb6bc2") |
| get_prompt_ids | Ran | exponentialml/text-to-video-finetuning/utils/dataset.py code served (permissive licence) · get_code("6f7c993202e44dda") |
| is_video | Ran | exponentialml/text-to-video-finetuning/models/unet_3d_blocks.py code served (permissive licence) · get_code("c1232f9d1e7794e4") |
| min_res | Ran | exponentialml/text-to-video-finetuning/utils/bucketing.py code served (permissive licence) · get_code("40775842d99a25f9") |
| normalize_input | Ran | exponentialml/text-to-video-finetuning/utils/dataset.py code served (permissive licence) · get_code("dbfc36f5a5a5a1fd") |
| read_caption_file | Ran | exponentialml/text-to-video-finetuning/utils/dataset.py code served (permissive licence) · get_code("00090fcbdd45d352") |
| up_down_bucket | Ran | exponentialml/text-to-video-finetuning/utils/bucketing.py code served (permissive licence) · get_code("bec7ef14686681c0") |
| convert_unet_state_dict | Not yet run | exponentialml/text-to-video-finetuning/utils/convert_diffusers_to_original_ms_text_to_video.py code served (permissive licence) · get_code("754cea3454b8b8da") |
| inject_inferable_lora | Not yet run | exponentialml/text-to-video-finetuning/utils/lora.py code served (permissive licence) · get_code("df1f737e7e9ec97f") |
| inject_trainable_lora | Not yet run | exponentialml/text-to-video-finetuning/utils/lora.py code served (permissive licence) · get_code("da50048edb81475c") |
| inject_trainable_lora_extended | Not yet run | exponentialml/text-to-video-finetuning/utils/lora.py code served (permissive licence) · get_code("929d4d45fed75939") |
| inpaint_watermark | Not yet run | exponentialml/text-to-video-finetuning/utils/lama.py code served (permissive licence) · get_code("3c2dce662245aae1") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
This paper introduces ModelScopeT2V, a text-to-video synthesis model that evolves from a text-to-image synthesis model (i.e., Stable Diffusion). ModelScopeT2V incorporates spatio-temporal blocks to ensure consistent frame generation and smooth movement transitions. The model could adapt to varying frame numbers during training and inference, rendering it suitable for both image-text and video-text datasets. ModelScopeT2V brings together three components (i.e., VQGAN, a text encoder, and a denoising UNet), totally comprising 1.7 billion parameters, in which 0.5 billion parameters are dedicated to temporal capabilities. The model demonstrates superior performance over state-of-the-art methods across three evaluation metrics. The code and an online demo are available at \url{https://modelscope.cn/models/damo/text-to-video-synthesis/summary}.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2308.06571")
get_code_for_paper("2308.06571")
have("2308.06571")
Connect an agent — have() is free.