Ziwei Liu, Shiqi Yang, Jingkang Yang, Jiankun Zhang, Chenyang Si, Jiajun Liang, Yukang Cao, Yucheng Liao, Jiazhe Wei, Ken Li, Wende Tan, Z Cui, and 4 more
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Language remains an outlier in modern generative modeling: while images, video, and audio are increasingly modeled in continuous latent spaces, text generation still relies predominantly on discrete tokens. Existing continuous language models either inherit embedding spaces not designed for generation and decoding jointly, or compress autoencoded latents to make diffusion easier at the cost of token-level fidelity. We challenge this prevailing design compromise. Instead of simplifying the representation to accommodate the generative model, we preserve a high-capacity, decodable text latent and design the diffusion model to learn its distribution directly. We introduce AURORA-LM, a continuous-latent diffusion language model that separates the construction of a decodable text representation from the modeling of its generative distribution. To obtain such a representation, we use a Query-based Encoder-Decoder that organizes text into a high-capacity, prefix-aligned latent sequence. We then introduce a Block-causal Diffusion Transformer that learns the distribution of these full-width latents through flow matching, generating blocks from left to right while denoising the positions within each block in parallel. However, retaining a high-capacity latent representation for accurate token decoding also makes its distribution more challenging for the diffusion model to learn. AURORA-LM addresses this difficulty by restricting only the noisy-input pathway while retaining the full clean-latent prediction target, allowing the generative model to accommodate the full-width latent without reducing decoderfacing capacity. We further calibrate the noise-level distribution to the latent width, accounting for how the effective signal strength changes with representation dimensionality. Finally, we introduce self-trajectory consistency to bridge the gap between training on independently sampled noisy states and inference through iterative denoising. Across comprehensive comparisons, AURORA-LM achieves the strongest performance among the evaluated continuous and diffusion-based language models on OpenWebText free generation and XSum conditional summarization. Scaling to 1B parameters with approximately 1500 EFLOPs of total compute yields further gains and surpasses a larger publicly released latent-diffusion language model under a matched evaluation protocol. Our results demonstrate that continuous language generation can effectively bridge diffusion-based generative modeling and discrete token decoding through a high-capacity, causally structured, and decodable text representation. All experiments are conducted on Ascend NPUs.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2608.02602")
get_code_for_paper("2608.02602")
have("2608.02602")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2608.02602.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2608.02602)
Connect an agent — have() is free.