Jingyi Yu, Sibei Yang, Cheng Shi, Hanzhuo Huang, Lan Xu, Yufan Feng
We lifted 11 functions out of this paper's own repositories and ran 5 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| soolab/free-bloom | canonical | 5 of 11 |
| Function | Status | Where it lives |
|---|---|---|
| diffusion_step | Ran | soolab/free-bloom/freebloom/prompt_attention/ptp_utils.py code served (permissive licence) · get_code("a042902caede448b") |
| get_noise_pred_single | Ran | soolab/free-bloom/freebloom/util.py code served (permissive licence) · get_code("0b088617d5525751") |
| latent2image | Ran | soolab/free-bloom/freebloom/prompt_attention/ptp_utils.py code served (permissive licence) · get_code("e3c3bb87bbe9b4a7") |
| next_step | Ran | soolab/free-bloom/freebloom/util.py code served (permissive licence) · get_code("fb032d6fa7e6196d") |
| text_under_image | Ran | soolab/free-bloom/freebloom/prompt_attention/ptp_utils.py code served (permissive licence) · get_code("45af788f85d9d0c3") |
| get_down_block | Not yet run | soolab/free-bloom/freebloom/models/unet_blocks.py code served (permissive licence) · get_code("a362e5b7a1be4609") |
| get_matrix | Not yet run | soolab/free-bloom/freebloom/prompt_attention/seq_aligner.py code served (permissive licence) · get_code("d167441d997c52ce") |
| get_matrix | Not yet run | soolab/free-bloom/freebloom/prompt_attention/seq_aligner.py code served (permissive licence) · get_code("81db21bf982ca0fe") |
| get_traceback_matrix | Not yet run | soolab/free-bloom/freebloom/prompt_attention/seq_aligner.py code served (permissive licence) · get_code("beb8f6a30678a4d9") |
| get_up_block | Not yet run | soolab/free-bloom/freebloom/models/unet_blocks.py code served (permissive licence) · get_code("01feceecb56a7ffb") |
| init_prompt | Not yet run | soolab/free-bloom/freebloom/util.py code served (permissive licence) · get_code("a0a4d94af120c56f") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Text-to-video is a rapidly growing research area that aims to generate a semantic, identical, and temporal coherence sequence of frames that accurately align with the input text prompt. This study focuses on zero-shot text-to-video generation considering the data- and cost-efficient. To generate a semantic-coherent video, exhibiting a rich portrayal of temporal semantics such as the whole process of flower blooming rather than a set of "moving images", we propose a novel Free-Bloom pipeline that harnesses large language models (LLMs) as the director to generate a semantic-coherence prompt sequence, while pre-trained latent diffusion models (LDMs) as the animator to generate the high fidelity frames. Furthermore, to ensure temporal and identical coherence while maintaining semantic coherence, we propose a series of annotative modifications to adapting LDMs in the reverse process, including joint noise sampling, step-aware attention shift, and dual-path interpolation. Without any video data and training requirements, Free-Bloom generates vivid and high-quality videos, awe-inspiring in generating complex scenes with semantic meaningful frame sequences. In addition, Free-Bloom is naturally compatible with LDMs-based extensions.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2309.14494")
get_code_for_paper("2309.14494")
have("2309.14494")
Connect an agent — have() is free.