Sergey Tulyakov, Yu Wu, Jian Ren, Kyle Olszewski, Ye Zhu, Yan Yan
We lifted 5 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| L-YeZhu/CDCD | canonical | 1 of 1 |
| l-yezhu/cdcd | — | 2 of 4 |
| Function | Status | Where it lives |
|---|---|---|
| alpha_schedule | Ran | l-yezhu/cdcd/synthesis/modeling/transformers/diffusion_d2m.py pointer only (licence: NONE) · get_code("84faa1a3b9691991") |
| beat_scores | Ran | L-YeZhu/CDCD/inference/inference_aist.py pointer only (licence: NONE) · get_code("ff2d0b5dcf33853c") |
| shift_time | Ran | l-yezhu/cdcd/synthesis/modeling/transformers/diffusion_d2m.py pointer only (licence: NONE) · get_code("4e22bf5a718f0d6a") |
| DiffusionTransformer | Not yet run | l-yezhu/cdcd/synthesis/modeling/transformers/diffusion_d2m.py pointer only (licence: NONE) · get_code("756f507ff7a03064") |
| instantiate_from_config | Not yet run | l-yezhu/cdcd/synthesis/modeling/transformers/diffusion_d2m.py pointer only (licence: NONE) · get_code("178128dbd7d43112") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Diffusion probabilistic models (DPMs) have become a popular approach to conditional generation, due to their promising results and support for cross-modal synthesis. A key desideratum in conditional synthesis is to achieve high correspondence between the conditioning input and generated output. Most existing methods learn such relationships implicitly, by incorporating the prior into the variational lower bound. In this work, we take a different route-we explicitly enhance input-output connections by maximizing their mutual information. To this end, we introduce a Conditional Discrete Contrastive Diffusion (CDCD) loss and design two contrastive diffusion mechanisms to effectively incorporate it into the denoising process, combining the diffusion training and contrastive learning for the first time by connecting it with the conventional variational objectives. We demonstrate the efficacy of our approach in evaluations with diverse multimodal conditional synthesis tasks: dance-to-music generation, text-to-image synthesis, as well as class-conditioned image synthesis. On each, we enhance the inputoutput correspondence and achieve higher or competitive general synthesis quality. Furthermore, the proposed approach improves the convergence of diffusion models, reducing the number of required diffusion steps by more than 35% on two benchmarks, significantly increasing the inference speed.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2206.07771")
get_code_for_paper("2206.07771")
have("2206.07771")
Connect an agent — have() is free.