SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2212.09462 · NeurIPS · 2023

Latent Diffusion for Language Generation

Justin Lovelace, Varsha Chao, Wan Eliot, Shekhtman Kilian, Q Weinberger

arXiv · PDF · Open in the Atlas

Code that ran

10 of 10 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (1 of them constructed an object rather than computing a result; 3 honoured the contract we drafted, 1 violated it and 1 has no contract verdict; 5 failures of our instrument; 0 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2212.09462.json · badge: /api/ran/2212.09462.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
justinlovelace/latent-diffusion-for-language canonical (classifier label)
classifier basis: classifier_2025
10 ran · 0 no recorded run
FunctionStatusWhere it lives
GaussianDiffusion Ran
ran on a synthesized input · constructed an object; no result was computed
justinlovelace/latent-diffusion-for-language/diffusion/text_denoising_diffusion.py
code served (permissive licence) · get_code("e7b1995f962c95ba")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, GaussianDiffusion: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from functools import partial

torch.manual_seed(0)

class DummyDiffusionModel(nn.Module):
    """Minimal diffusion model with required attributes"""
    def __init__(self):
        super().__init__()
        self.class_conditional = False
        self.class_unconditional_prob = 0.0
        self.latent_dim = 4
        self.self_condition = False
        self.linear = nn.Linear(4, 4)
    
    def forward(self, x, t, cond=None, self_cond=None):
        return self.linear(x)

model = DummyDiffusionModel()

ARGS = {
    'model': model,
    'max_seq_len': 16,
    'sampling_timesteps': 250,
    'loss_type': 'l1',
    'objective': 'pred_noise',
    'train_schedule': 'cosine',
    'sampling_schedule': None,
    'scale': 1.0,
    'sampler': 'ddpm',
    'train_prob_self_cond': 0.5,
    'seq2seq_unconditional_prob': 0.1,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
einops, numpy, torch, tqdm — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type GaussianDiffusion — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a GaussianDiffusion output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
alpha_to_shifted_log_snr Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
justinlovelace/latent-diffusion-for-language/diffusion/text_denoising_diffusion.py
code served (permissive licence) · get_code("d5b5906bd8d6f099")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (1, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# Construct a tensor for alpha parameter
# alpha should be values between 0 and 1 for the function to work meaningfully
alpha = torch.tensor([[0.3, 0.5, 0.7, 0.9]], dtype=torch.float32)

# Construct scale parameter (a scalar that will be converted with np.log)
scale = 1.0

ARGS = {
    "alpha": alpha,
    "scale": scale,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (1, 4) — shape (1, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
beta_linear_schedule Ran
ran, honoured the contract we drafted
justinlovelace/latent-diffusion-for-language/diffusion/text_denoising_diffusion.py
code served (permissive licence) · get_code("e619cd65a90a008c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Analyzing the function signature:
# def beta_linear_schedule(t, clip_min=1e-09):
# 
# Parameters:
# - t: used in the expression (-0.0001 - 10 * t ** 2), so should be a tensor or numeric value
# - clip_min: default value provided, used in .clamp(min=clip_min, max=1.0)
#
# The function computes torch.exp(-0.0001 - 10 * t ** 2).clamp(min=clip_min, max=1.0)
# So t should be a numeric value or tensor that can be squared and used in arithmetic

# Create a small tensor for t
t = torch.tensor(0.5)

# clip_min has a default, but we can override it
clip_min = 1e-09

ARGS = {
    "t": t,
    "clip_min": clip_min
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
cosine_schedule Ran
ran; our contract draft was wrong, not the code
justinlovelace/latent-diffusion-for-language/diffusion/text_denoising_diffusion.py
code served (permissive licence) · get_code("59dbcc3beaee75d7")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 1, dtype family float, shape (5,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import math

torch.manual_seed(0)

# The function takes:
# - t: a tensor that gets operations applied (cos, power, clamp)
# - start: scalar (default 0)
# - end: scalar (default 1)
# - tau: scalar (default 1)
# - clip_min: scalar (default 1e-09)

# Create a small tensor for t
t = torch.linspace(0, 1, 5)

ARGS = {
    "t": t,
    "start": 0,
    "end": 1,
    "tau": 1,
    "clip_min": 1e-09
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 1, dtype family float, shape (5,) — shape (5,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
log Ran
ran; our contract draft was wrong, not the code
justinlovelace/latent-diffusion-for-language/diffusion/text_denoising_diffusion.py
code served (permissive licence) · get_code("d93be7ef30d67f1e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# The function takes a tensor `t` and an optional `eps` parameter
# It clamps the tensor to a minimum value and takes the log
# We need to provide a valid tensor argument

t = torch.rand(2, 4)  # Small tensor with batch size 2, dim 4

ARGS = {
    "t": t,
    "eps": 1e-12
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
log_snr_to_alpha Ran
ran, honoured the contract we drafted
justinlovelace/latent-diffusion-for-language/diffusion/text_denoising_diffusion.py
code served (permissive licence) · get_code("8697654fdac09b54")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# log_snr_to_alpha takes a log_snr tensor and applies sigmoid to it
# The function expects a tensor as input
log_snr = torch.randn(2, 4)

ARGS = {
    "log_snr": log_snr
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
right_pad_dims_to Ran
ran, violated the contract we drafted
justinlovelace/latent-diffusion-for-language/diffusion/text_denoising_diffusion.py
code served (permissive licence) · get_code("8cd8e6547ade35f4")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 4, dtype family float, shape (2, 4, 1, 1) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# x is accessed for x.ndim, so it needs to be a tensor
x = torch.randn(2, 4, 3, 5)

# t is accessed for t.ndim, t.shape, and view(), so it needs to be a tensor
t = torch.randn(2, 4)

ARGS = {
    'x': x,
    't': t,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 4, dtype family float, shape (2, 4, 1, 1) — shape (2, 4, 1, 1) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
sigmoid_schedule Ran
ran; our contract draft was wrong, not the code
justinlovelace/latent-diffusion-for-language/diffusion/text_denoising_diffusion.py
code served (permissive licence) · get_code("cf03aea4a2411cc6")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 1, dtype family float, shape (2,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Analyze the function:
# - t: used in arithmetic operations, needs to be a tensor or scalar
# - start: scalar used in arithmetic (default -3)
# - end: scalar used in arithmetic (default 3)
# - tau: scalar used in division (default 1)
# - clamp_min: scalar used in clamp_ (default 1e-09)

# The function performs sigmoid scheduling calculations
# t is the time parameter, can be a tensor or scalar

# Create a small tensor for t (time steps)
t = torch.tensor([0.0, 0.5])

ARGS = {
    "t": t,
    "start": -3,
    "end": 3,
    "tau": 1,
    "clamp_min": 1e-09
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 1, dtype family float, shape (2,) — shape (2,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
simple_linear_schedule Ran
ran, honoured the contract we drafted
justinlovelace/latent-diffusion-for-language/diffusion/text_denoising_diffusion.py
code served (permissive licence) · get_code("adb4575b401b32ab")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Analyze the function:
# def simple_linear_schedule(t, clip_min=1e-09):
#     return (1 - t).clamp(min=clip_min)
# 
# Parameters:
# - t: used in arithmetic (1 - t) and .clamp() call, so it's a tensor or scalar
# - clip_min: used as keyword argument to .clamp(), so it's a numeric value (float)

# Create a small tensor for t
t = torch.tensor(0.5)

# clip_min has a default value of 1e-09, we can use the default or provide a value
clip_min = 1e-09

ARGS = {
    "t": t,
    "clip_min": clip_min,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
time_to_alpha Ran
ran; our contract draft was wrong, not the code
justinlovelace/latent-diffusion-for-language/diffusion/text_denoising_diffusion.py
code served (permissive licence) · get_code("13dc3dd849ea08a9")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 1, dtype family float, shape (2,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# Define alpha_schedule as a callable that takes t and returns alpha values
class AlphaSchedule:
    def __call__(self, t):
        # Simple linear schedule: alpha decreases from 1 to 0 as t goes from 0 to 1
        # Return values in (0, 1) range suitable for alpha
        return torch.sigmoid(torch.tensor([0.5] * t.shape[0] if isinstance(t, torch.Tensor) else [0.5]))

def simple_alpha_schedule(t):
    # t could be a tensor or scalar
    # Return alpha values in (0, 1) range
    if isinstance(t, torch.Tensor):
        return torch.full_like(t, 0.7, dtype=torch.float32)
    else:
        return torch.tensor(0.7, dtype=torch.float32)

# Create test tensors
t = torch.tensor([0.2, 0.5], dtype=torch.float32)  # batch of 2 time values
alpha_schedule = simple_alpha_schedule
scale = torch.tensor(1.0)

ARGS = {
    "t": t,
    "alpha_schedule": alpha_schedule,
    "scale": scale
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 1, dtype family float, shape (2,) — shape (2,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block

Also present in other papers' code

Of 10 samples lifted from this paper's linked repositories, 10 carry a function identity and 0 do not. Of the 10 distinct functions with an identity, 3 are also present in other papers whose code we hold (1 with at least one paper dated earlier, 0 the same month, 3 later) and 7 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
right_pad_dims_to
brick of this paper's assembly: helper
4 other papers
2 dated earlier, 2 later · 2 only as an assembly brick
2208.04202 (2022-08)
Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning
2209.15571 (2022-09)
Building Normalizing Flows with Stochastic Interpolants
2306.08276 (2023-06)
TryOnDiffusion: A Tale of Two UNets
and 1 more not named here
log
brick of this paper's assembly: helper
2 other papers
2 later
2408.04220 (2024-08)
Diffusion Guided Language Modeling
2602.20528 (2026-02)
Stop-Think-AutoRegress: Language Modeling with Latent Diffusion Planning
cosine_schedule
brick of this paper's assembly: scheduler
1 other paper
1 later
2408.04220 (2024-08)
Diffusion Guided Language Modeling

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Diffusion models have achieved great success in modeling continuous data modalities such as images, audio, and video, but have seen limited use in discrete domains such as language. Recent attempts to adapt diffusion to language have presented diffusion as an alternative to existing pretrained language models. We view diffusion and existing language models as complementary. We demonstrate that encoder-decoder language models can be utilized to efficiently learn high-quality language autoencoders. We then demonstrate that continuous diffusion models can be learned in the latent space of the language autoencoder, enabling us to sample continuous latent representations that can be decoded into natural language with the pretrained decoder. We validate the effectiveness of our approach for unconditional, class-conditional, and sequence-to-sequence language generation. We demonstrate across multiple diverse data sets that our latent language diffusion models are significantly more effective than previous diffusion language models. Our code is available at https://github.com/justinlovelace/ latent-diffusion-for-language.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2212.09462")
get_code_for_paper("2212.09462")
have("2212.09462")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2212.09462.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2212.09462.svg)](https://syntology.ai/paper/2212.09462)

Connect an agent — have() is free.