SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2211.17192 · ICML · 2023

Fast Inference from Transformers via Speculative Decoding

Yossi Matias, Yaniv Leviathan, Matan Kalman

arXiv · PDF · Open in the Atlas

Code that ran

15 of 21 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (1 of them constructed an object rather than computing a result; 2 honoured the contract we drafted, 0 violated it and 4 have no contract verdict; 9 failures of our instrument; 6 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2211.17192.json · badge: /api/ran/2211.17192.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
feifeibear/llmspeculativesampling community
Papers with Code archive
5 ran · 3 no recorded run
lucidrains/speculative-decoding community
Papers with Code archive
4 ran · 0 no recorded run
smart-lty/parallelspeculativedecoding community
Papers with Code archive
3 ran · 3 no recorded run
ericlbuehler/mistral.rs community (classifier label)
classifier basis not recorded on the link
3 ran · 0 no recorded run
FunctionStatusWhere it lives
KVCacheModel Ran
ran on a synthesized input · constructed an object; no result was computed
feifeibear/llmspeculativesampling/sampling/speculative_sampling.py
code served (permissive licence) · get_code("b8211819644d71f6")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, KVCacheModel: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from transformers.models.bloom.modeling_bloom import BloomForCausalLM
from torch.nn import functional as F

torch.manual_seed(0)

# Build a minimal model that mimics BloomForCausalLM behavior
class MockBloomModel(nn.Module):
    def __init__(self, vocab_size=256, hidden_size=32, num_layers=2):
        super().__init__()
        self.vocab_size = vocab_size
        self.hidden_size = hidden_size
        self.num_layers = num_layers
        
        # Minimal embeddings and linear layers
        self.embed = nn.Embedding(vocab_size, hidden_size)
        self.lm_head = nn.Linear(hidden_size, vocab_size)
    
    def forward(self, input_ids, past_key_values=None, use_cache=True):
        # input_ids shape: (batch, seq_len)
        batch_size, seq_len = input_ids.shape
        
        # Embed tokens
        hidden_states = self.embed(input_ids)  # (batch, seq_len, hidden_size)
        
        # Compute logits
        logits = self.lm_head(hidden_states)  # (batch, seq_len, vocab_size)
        
        # Generate KV cache (mock: simplified representation)
        # Each layer has (k, v) tuples
        if use_cache:
            if past_key_values is None:
                # First pass: create cache for all tokens
                new_past_key_values = []
                for _ in range(self.num_layers):
                    k = torch.randn(batch_size, hidden_size, seq_len, hidden_size // 8)
                    v = torch.randn(batch_size, hidden_size, seq_len, hidden_size // 8)
                    new_past_key_values.append((k, v))
            else:
                # Subsequent passes: append to cache
                new_past_key_values = []
                for layer_idx in range(self.num_layers):
                    k_prev, v_prev = past_key_values[layer_idx]
                    # Append new token's k,v
                    k_new = torch.randn(batch_size, hidden_size, k_prev.shape[2] + seq_len, hidden_size // 8)
                    v_new = torch.randn(batch_size, hidden_size, v_prev.shape[2] + seq_len, hidden_size // 8)
                    new_past_key_values.append((k_new, v_new))
        else:
            new_past_key_values = None
        
        # Return output object with logits and past_key_values
        class Output:
            pass
        
        output = Output()
        output.logits = logits
        output.past_key_values = new_past_key_values
        return output

# Create model instance
model = MockBloomModel(vocab_size=256, hidden_size=32, num_layers=2)

# Define ARGS dictionary
ARGS = {
    'model': model,
    'temperature': 1.0,
    'top_k': 10,
    'top_p': 0.9,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type KVCacheModel — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a KVCacheModel output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
detect_dataframe Ran
ran on a synthesized input
ericlbuehler/mistral.rs/mistralrs-code-exec/python/executor.py
code served (permissive licence) · get_code("4fa98a77cb95f018")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import pandas as pd

# Create a pandas DataFrame for testing
df = pd.DataFrame({
    'col1': [1, 2, 3],
    'col2': [4.0, 5.0, 6.0]
})

# Create a pandas Series for testing
series = pd.Series([10, 20, 30], name='test_series')

ARGS = {
    'obj': df
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
pandas — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
detect_pil_image Ran
ran on a synthesized input
ericlbuehler/mistral.rs/mistralrs-code-exec/python/executor.py
code served (permissive licence) · get_code("905226f05e395d5f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import io
import base64
from PIL import Image
import numpy as np

# Create a simple PIL Image object
img_array = np.random.randint(0, 256, (10, 10, 3), dtype=np.uint8)
pil_image = Image.fromarray(img_array, mode='RGB')

# Define the arguments dictionary
ARGS = {
    'obj': pil_image
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
PIL, numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
extract_outputs Ran
ran on a synthesized input
ericlbuehler/mistral.rs/mistralrs-code-exec/python/executor.py
code served (permissive licence) · get_code("c512e543f906b1ca")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import os
import tempfile
import shutil

# Create a temporary work directory with test files
temp_dir = tempfile.mkdtemp()

# Create some test files that extract_outputs can read
# Test 1: A text file (json format)
json_file = os.path.join(temp_dir, "output.json")
with open(json_file, 'w') as f:
    f.write('{"result": "success"}')

# Test 2: A markdown file
md_file = os.path.join(temp_dir, "report.md")
with open(md_file, 'w') as f:
    f.write('# Report\nThis is a test.')

# Test 3: A CSV file
csv_file = os.path.join(temp_dir, "data.csv")
with open(csv_file, 'w') as f:
    f.write('name,value\ntest,123')

# Test 4: A binary file (png)
png_file = os.path.join(temp_dir, "image.png")
with open(png_file, 'wb') as f:
    f.write(b'\x89PNG\r\n\x1a\n' + b'\x00' * 100)  # Minimal PNG header

# Create the outputs list with entries referencing these files
outputs = [
    {'name': 'output.json', 'format': 'json'},
    {'name': 'report.md'},  # format will be inferred from extension
    {'name': 'data.csv', 'format': 'csv'},
    {'name': 'image.png'},  # binary format
    {'name': 'nonexistent.txt'},  # will produce 'not produced' error
    {'name': None},  # will produce 'missing name' error
]

# Store the temp directory for cleanup later if needed
_temp_dir = temp_dir

ARGS = {
    'outputs': outputs
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
find_first_true_index Ran
ran, honoured the contract we drafted
lucidrains/speculative-decoding/speculative_decoding/speculative_decoding.py
code served (permissive licence) · get_code("ef39dd586a82a156")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 1, dtype family int, shape (2,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create a small boolean tensor
# Shape: (2, 4) - batch size 2, dimension 4
bool_tensor = torch.tensor([
    [True, False, False, True],
    [False, False, True, False]
], dtype=torch.bool)

ARGS = {
    "bool_tensor": bool_tensor,
    "dim": -1
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 1, dtype family int, shape (2,) — shape (2,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
max_fn Ran
ran; our contract draft was wrong, not the code
feifeibear/llmspeculativesampling/sampling/speculative_sampling.py
code served (permissive licence) · get_code("d5c48d2733f29ebb")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# The function max_fn takes a single parameter x
# Looking at the body:
# - x is used with torch.where, torch.zeros_like
# - x is compared with 0 (x > 0)
# - torch.sum is called on x with dim=1, keepdim=True
# - This implies x should be a 2D tensor (batch, features)
# So x should be a tensor with shape (batch_size, feature_dim)

x = torch.randn(2, 4)  # batch size 2, feature dim 4

ARGS = {
    "x": x
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
norm_logits Ran
ran; our contract draft was wrong, not the code
feifeibear/llmspeculativesampling/sampling/speculative_sampling.py
code served (permissive licence) · get_code("c9dd1e87e985a33c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn.functional as F

torch.manual_seed(0)

# Create a small batch of logits
# shape: (batch_size=2, vocab_size=4)
logits = torch.randn(2, 4)

# Temperature parameter
temperature = 0.7

# Top-k parameter
top_k = 2

# Top-p parameter
top_p = 0.9

ARGS = {
    "logits": logits,
    "temperature": temperature,
    "top_k": top_k,
    "top_p": top_p,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
norm_logits Ran
ran; our contract draft was wrong, not the code
smart-lty/parallelspeculativedecoding/src/engine.py
pointer only (licence: NONE) · get_code("ea7422602cf8d39a")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, smart-lty/parallelspeculativedecoding when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

safe_div Ran
ran, honoured the contract we drafted
lucidrains/speculative-decoding/speculative_decoding/speculative_decoding.py
code served (permissive licence) · get_code("806a4b61f4218c4b")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

ARGS = {
    "num": 5.0,
    "den": 2.0,
    "eps": 1e-10
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
sample Ran
ran; our contract draft was wrong, not the code
feifeibear/llmspeculativesampling/sampling/speculative_sampling.py
code served (permissive licence) · get_code("d433f14ebfd69413")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 1, dtype family int, shape (1,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# The function expects:
# - probs: torch.Tensor that can be passed to torch.multinomial
# - num_samples: int (optional, defaults to 1)

# torch.multinomial expects a 1D or 2D tensor of probabilities
# Create a valid probability distribution (sums to 1)
probs = torch.tensor([0.1, 0.3, 0.4, 0.2])

ARGS = {
    "probs": probs,
    "num_samples": 1
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 1, dtype family int, shape (1,) — shape (1,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
sample Ran
ran; our contract draft was wrong, not the code
smart-lty/parallelspeculativedecoding/src/engine.py
pointer only (licence: NONE) · get_code("ffa10e3a85932ece")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, smart-lty/parallelspeculativedecoding when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

speculative_decoding Ran
ran; our contract draft was wrong, not the code
lucidrains/speculative-decoding/speculative_decoding/speculative_decoding.py
code served (permissive licence) · get_code("ce874c7fa5d31d78")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from torch import Tensor

torch.manual_seed(0)

# Build minimal nn.Module for net and small_net
# They need to:
# 1. Accept (out, seq_start_pos=..., cache=..., return_cache=True)
# 2. Return (logits, cache) where logits shape is (batch, seq_len, vocab_size)
# 3. cache is a tuple of (cached_kv, None) where cached_kv has shape compatible with indexing

class MinimalTransformer(nn.Module):
    def __init__(self, vocab_size=16, hidden_dim=8):
        super().__init__()
        self.vocab_size = vocab_size
        self.hidden_dim = hidden_dim
        self.embedding = nn.Embedding(vocab_size, hidden_dim)
        self.linear = nn.Linear(hidden_dim, vocab_size)
    
    def forward(self, out, seq_start_pos=None, cache=None, return_cache=False):
        # out shape: (batch, seq_len)
        batch_size, seq_len = out.shape
        
        # Embed and project to logits
        embedded = self.embedding(out)  # (batch, seq_len, hidden_dim)
        logits = self.linear(embedded)  # (batch, seq_len, vocab_size)
        
        # Create a dummy cache with shape (batch, num_heads, seq_len, head_dim)
        # We need it to be indexable by batch_range and seq_offset_indices
        num_heads = 2
        head_dim = 4
        cached_kv = torch.randn(batch_size, num_heads, seq_len, head_dim)
        
        if return_cache:
            return logits, (cached_kv, None)
        return logits

# Create instances
batch_size = 2
vocab_size = 16
prompt_len = 5
seq_len = 10

net = MinimalTransformer(vocab_size=vocab_size, hidden_dim=8)
small_net = MinimalTransformer(vocab_size=vocab_size, hidden_dim=8)

# Create prompt tensor (batch, seq_len) with token ids
prompt = torch.randint(0, vocab_size, (batch_size, prompt_len), dtype=torch.long)

ARGS = {
    'net': net,
    'small_net': small_net,
    'prompt': prompt,
    'seq_len': seq_len,
    'gamma': 3,
    'temperature': 1.0,
    'filter_thres': 0.9,
    'lenience': 1.0,
    'pad_id': 0,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
einops, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
top_k Ran
ran; our contract draft was wrong, not the code
lucidrains/speculative-decoding/speculative_decoding/speculative_decoding.py
code served (permissive licence) · get_code("3c61d89a5e42a3d3")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import math
import torch

torch.manual_seed(0)

# Create a small logits tensor with shape (batch=2, vocab=4)
logits = torch.randn(2, 4)

# Set threshold parameter
thres = 0.9

# Define ARGS dict with all parameters
ARGS = {
    'logits': logits,
    'thres': thres
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
top_k_top_p_filter Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
feifeibear/llmspeculativesampling/sampling/speculative_sampling.py
code served (permissive licence) · get_code("a391388e6c7e3baa")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch.nn import functional as F

torch.manual_seed(0)

# Create a small logits tensor with shape (batch=2, vocab=4)
logits = torch.randn(2, 4)

# Define arguments for the function
ARGS = {
    "logits": logits,
    "top_k": 2,
    "top_p": 0.9,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
top_k_top_p_filter Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
smart-lty/parallelspeculativedecoding/src/engine.py
pointer only (licence: NONE) · get_code("025c3469ca2fb4c3")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, smart-lty/parallelspeculativedecoding when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Decoder Not yet run
unverified (harvested, no recorded run)
feifeibear/llmspeculativesampling/sampling/speculative_sampling.py
code served (permissive licence) · get_code("23abd23ad9a26aaf")
How it ran

no invocation recorded.

Decoding Not yet run
unverified (harvested, no recorded run)
smart-lty/parallelspeculativedecoding/src/engine.py
pointer only (licence: NONE) · get_code("559f754be1db32cf")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, smart-lty/parallelspeculativedecoding when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

KVCacheModel Not yet run
unverified (harvested, no recorded run)
smart-lty/parallelspeculativedecoding/src/engine.py
pointer only (licence: NONE) · get_code("c09397f1dfae4d19")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, smart-lty/parallelspeculativedecoding when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

Singleton Not yet run
unverified (harvested, no recorded run)
feifeibear/llmspeculativesampling/sampling/speculative_sampling.py
code served (permissive licence) · get_code("49c7d0a8e1eaff77")
How it ran

no invocation recorded.

seed_everything Not yet run
unverified (harvested, no recorded run)
smart-lty/parallelspeculativedecoding/src/engine.py
pointer only (licence: NONE) · get_code("79fc1fb911ac4bf9")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, smart-lty/parallelspeculativedecoding when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. No invocation recorded.

speculative_sampling Not yet run
unverified (harvested, no recorded run)
feifeibear/llmspeculativesampling/sampling/speculative_sampling.py
code served (permissive licence) · get_code("eb5f131d8785dd2d")
How it ran

no invocation recorded.

Also present in other papers' code

Of 21 samples lifted from this paper's linked repositories, 21 carry a function identity and 0 do not. Of the 20 distinct functions with an identity, 3 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 3 later) and 17 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
norm_logits
brick of this paper's assembly: helper
1 other paper
1 later
2408.11850 (2024-08)
PEARL: PARALLEL SPECULATIVE DECODING WITH ADAPTIVE DRAFT LENGTH
sample
brick of this paper's assembly: sampler
1 other paper
1 later
2408.11850 (2024-08)
PEARL: PARALLEL SPECULATIVE DECODING WITH ADAPTIVE DRAFT LENGTH
top_k_top_p_filter
brick of this paper's assembly: helper, sampler
1 other paper
1 later
2408.11850 (2024-08)
PEARL: PARALLEL SPECULATIVE DECODING WITH ADAPTIVE DRAFT LENGTH

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Inference from large autoregressive models like Transformers is slow -decoding K tokens takes K serial runs of the model. In this work we introduce speculative decoding -an algorithm to sample from autoregressive models faster without any changes to the outputs, by computing several tokens in parallel. At the heart of our approach lie the observations that (1) hard language-modeling tasks often include easier subtasks that can be approximated well by more efficient models, and (2) using speculative execution and a novel sampling method, we can make exact decoding from the large models faster, by running them in parallel on the outputs of the approximation models, potentially generating several tokens concurrently, and without changing the distribution. Our method can accelerate existing off-the-shelf models without retraining or architecture changes. We demonstrate it on T5-XXL and show a 2X-3X acceleration compared to the standard T5X implementation, with identical outputs.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2211.17192")
get_code_for_paper("2211.17192")
have("2211.17192")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2211.17192.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2211.17192.svg)](https://syntology.ai/paper/2211.17192)

Connect an agent — have() is free.