We lifted 3 functions out of this paper's own repositories and ran 3 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| lingxiaoshawn/usd3 | canonical | 3 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| sample_bernoulli | Ran | lingxiaoshawn/usd3/discrete_diffusion.py code served (permissive licence) · get_code("b7292183c8ee8d08") |
| sample_categorical | Ran | lingxiaoshawn/usd3/discrete_diffusion.py code served (permissive licence) · get_code("3b3f251a40b42cb6") |
| sample_uniform_categorical | Ran | lingxiaoshawn/usd3/discrete_diffusion.py code served (permissive licence) · get_code("3de7a59d8f5a3e18") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Discrete diffusion models have seen a surge of attention with applications on naturally discrete data such as language and graphs. Although discrete-time discrete diffusion has been established for a while, only recently Campbell et al. (2022) introduced the first framework for continuous-time discrete diffusion. However, their training and sampling processes differ significantly from the discrete-time version, necessitating nontrivial approximations for tractability. In this paper, we first present a series of mathematical simplifications of the variational lower bound that enable more accurate and easy-to-optimize training for discrete diffusion. In addition, we derive a simple formulation for backward denoising that enables exact and accelerated sampling, and importantly, an elegant unification of discrete-time and continuous-time discrete diffusion. Thanks to simpler analytical formulations, both forward and now also backward probabilities can flexibly accommodate any noise distribution, including different noise distributions for multi-element objects. Experiments show that our proposed USD3 (for Unified Simplified Discrete Denoising Diffusion) outperform all SOTA baselines on established datasets. We open-source our unified code at https://github.com/LingxiaoShawn/USD3.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2402.03701")
get_code_for_paper("2402.03701")
have("2402.03701")
Connect an agent — have() is free.