SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2004.03991 · ICML · 2020

Learning Discrete Structured Representations by Adversarially Maximizing Mutual Information

Karl Stratos, Sam Wiseman

arXiv · PDF · Open in the Atlas

Code that ran

22 of 27 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (6 of them constructed an object rather than computing a result; 3 honoured the contract we drafted, 0 violated it and 6 have no contract verdict; 13 failures of our instrument; 5 no recorded run), measured 2026-11-22; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2004.03991.json · badge: /api/ran/2004.03991.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
karlstratos/ammi canonical (classifier label)
classifier basis: classifier_2025
22 ran · 5 no recorded run
FunctionStatusWhere it lives
ArticlePairDataset Ran
ran on a synthesized input · constructed an object; no result was computed
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("bc38d49d94d3dd4b")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ArticlePairDataset: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create sample dict_pairs
# Each pair is (article_X, article_Y) where each article is a dict of {index: weight}
dict_pairs = [
    (
        {0: 0.5, 2: 0.3},  # article_X: sparse vector representation
        {1: 0.8, 3: 0.2}   # article_Y: sparse vector representation
    ),
    (
        {1: 0.4, 3: 0.6},
        {0: 0.7, 2: 0.1}
    )
]

vocab_size = 4

ARGS = {
    "dict_pairs": dict_pairs,
    "vocab_size": vocab_size
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ArticlePairDataset — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ArticlePairDataset output the recorded ndim 3 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ArticlePairs Ran
ran on a synthesized input · constructed an object; no result was computed
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("dd979f2cfad3adfa")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ArticlePairs: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import pickle
import tempfile
import os

torch.manual_seed(0)

# Create a temporary pickle file with mock data
temp_dir = tempfile.mkdtemp()
file_path = os.path.join(temp_dir, "article_pairs.pkl")

# Create mock article pair data
# Each article is represented as a dict mapping vocab indices to weights
vocab_size = 4
vocab = {0: "word0", 1: "word1", 2: "word2", 3: "word3"}

# Create sample article pairs (sparse representation as dicts)
train_pairs = [
    ({0: 0.5, 1: 0.3}, {2: 0.4, 3: 0.6}),
    ({1: 0.7, 3: 0.2}, {0: 0.1, 2: 0.9}),
]

val_pairs = [
    ({0: 0.6, 2: 0.4}, {1: 0.5, 3: 0.5}),
]

test_pairs = [
    ({1: 0.8, 3: 0.2}, {0: 0.3, 2: 0.7}),
]

# Pickle the data
with open(file_path, 'wb') as f:
    pickle.dump((train_pairs, val_pairs, test_pairs, vocab), f)

ARGS = {
    "file_path": file_path
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ArticlePairs — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ArticlePairs output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
FF Ran
ran on a synthesized input · constructed an object; no result was computed
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("5254fe49108bccef")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, FF: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    'dim_input': 4,
    'dim_hidden': 8,
    'dim_output': 3,
    'num_layers': 2,
    'activation': 'relu',
    'dropout_rate': 0.1,
    'layer_norm': True,
    'residual_connection': False,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type FF — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a FF output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Logger Ran
ran on a synthesized input · constructed an object; no result was computed
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("62aafeb07c870d8c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Logger: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import math
import os
import statistics as stat
import sys
import tempfile

ARGS = {
    'log_path': os.path.join(tempfile.gettempdir(), 'test_logger.log'),
    'on': True
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Logger — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Logger output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Posterior Ran
ran on a synthesized input · constructed an object; no result was computed
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("a53a047167b58dcc")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Posterior: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    'num_features': 3,
    'markov_order': 2,
    'dim_input': 5,
    'num_layers': 2,
    'dim_hidden': 4,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Posterior — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Posterior output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Prior Ran
ran on a synthesized input · constructed an object; no result was computed
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("67356a959b93e551")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Prior: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    'num_features': 5,
    'markov_order': 2,
    'num_layers': 1,
    'dim_hidden': 8,
    'raw': False,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Prior — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Prior output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
compute_distance Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("87baeee1051adfd9")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 3) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create small test tensors
# X1: shape (2, 4) - 2 samples, 4 dimensions
# X2: shape (3, 4) - 3 samples, 4 dimensions
X1 = torch.randint(0, 2, (2, 4))
X2 = torch.randint(0, 2, (3, 4))

ARGS = {
    'X1': X1,
    'X2': X2,
    'distance_metric': 'hamming',
    'chunk_size': 1000,
    'binary': True
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 3) — shape (2, 3) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
compute_forward Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("dd7602f30bfb18b5")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 3, dtype family float, shape (2, 4, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import math

torch.manual_seed(0)

# Parameters for compute_forward
# P: [0,1]^{B x m x 2^(o+1)}
# We need to choose o such that 2^(o+1) is reasonable
# Let's use o=2, so 2^(o+1) = 8

B = 2  # batch size
m = 4  # sequence length
o = 2  # order parameter
P_size_last = 2 ** (o + 1)  # = 8

# Create P tensor with shape [B, m, 2^(o+1)]
# Values should be in [0,1] (probabilities)
P = torch.rand(B, m, P_size_last)

# quads can be None (will be computed) or precomputed
# Let's pass None to test the internal computation
quads = None

ARGS = {
    "P": P,
    "quads": quads,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 3, dtype family float, shape (2, 4, 4) — shape (2, 4, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
compute_hamming_distance Ran
ran; our contract draft was wrong, not the code
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("6391762af9ece0f9")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (6, 3) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create two binary tensors with distinct dimensions
# X1: shape (6, 4) - 6 samples, 4 features
# X2: shape (3, 4) - 3 samples, 4 features
X1 = torch.randint(0, 2, (6, 4))
X2 = torch.randint(0, 2, (3, 4))

ARGS = {
    "X1": X1,
    "X2": X2,
    "chunk_size": 2,
    "binary": True
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (6, 3) — shape (6, 3) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
compute_matching_accuracy Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("eb7e2974e8dff8a6")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Create a minimal encode_discrete function that processes tensors
def encode_discrete(X):
    # Simple encoding: convert to binary representation
    return (X > X.mean(dim=0, keepdim=True)).long()

# Create a minimal data loader (list of (X, Y) tuples)
# Use distinct sizes: 2 samples per batch, 4 features, different dimensions
eval_loader = [
    (torch.randn(2, 4), torch.randn(2, 4)),
    (torch.randn(2, 4), torch.randn(2, 4)),
]

# Device (CPU only)
device = torch.device('cpu')

# Define ARGS dict with all parameters of compute_matching_accuracy
ARGS = {
    'eval_loader': eval_loader,
    'device': device,
    'encode_discrete': encode_discrete,
    'distance_metric': 'hamming',
    'num_retrieve': 2,
    'chunk_size': 1,
    'binary': True,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
compute_retrieval_precision Ran
ran, honoured the contract we drafted
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("78810712770d61ac")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Create small mock data loaders
# Each loader is a list of (docs, labels) tuples
# docs: binary encodings of shape (batch_size, encoding_dim)
# labels: sparse multi-hot labels of shape (batch_size, num_labels)

# Training loader
train_docs_1 = torch.randint(0, 2, (2, 4)).float()
train_labels_1 = torch.zeros(2, 3)
train_labels_1[0, 0] = 1
train_labels_1[1, 1] = 1

train_docs_2 = torch.randint(0, 2, (2, 4)).float()
train_labels_2 = torch.zeros(2, 3)
train_labels_2[0, 2] = 1
train_labels_2[1, 0] = 1

train_loader = [
    (train_docs_1, train_labels_1),
    (train_docs_2, train_labels_2)
]

# Evaluation loader
eval_docs_1 = torch.randint(0, 2, (2, 4)).float()
eval_labels_1 = torch.zeros(2, 3)
eval_labels_1[0, 0] = 1
eval_labels_1[1, 1] = 1

eval_loader = [
    (eval_docs_1, eval_labels_1)
]

# Device (CPU only)
device = 'cpu'

# encode_discrete: optional function to encode docs
# Since the docs are already binary, we can use None or a simple identity function
encode_discrete = None

# distance_metric: 'hamming' or 'cosine'
distance_metric = 'hamming'

# num_retrieve: number of documents to retrieve
num_retrieve = 2

ARGS = {
    'train_loader': train_loader,
    'eval_loader': eval_loader,
    'device': device,
    'encode_discrete': encode_discrete,
    'distance_metric': distance_metric,
    'num_retrieve': num_retrieve
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
compute_topK_average_precision Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("32a95ddcdd120243")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create small tensors for encodings
# tgt_encodings: target encodings, shape (2, 4) - 2 samples, 4 dimensions
# src_encodings: source encodings, shape (3, 4) - 3 samples, 4 dimensions
tgt_encodings = torch.randint(0, 2, (2, 4))
src_encodings = torch.randint(0, 2, (3, 4))

# Create label lists
# tgt_label_lists: list of label sets for each target sample
tgt_label_lists = [
    ['label_a', 'label_b'],
    ['label_c']
]

# src_label_lists: list of label sets for each source sample
src_label_lists = [
    ['label_a'],
    ['label_b', 'label_d'],
    ['label_c', 'label_e']
]

# Number of retrievals to compute top-K for
num_retrieve = 2

ARGS = {
    'tgt_encodings': tgt_encodings,
    'tgt_label_lists': tgt_label_lists,
    'src_encodings': src_encodings,
    'src_label_lists': src_label_lists,
    'num_retrieve': num_retrieve,
    'distance_metric': 'hamming',
    'chunk_size': 100,
    'binary': True
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
compute_viterbi Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("a4880a127313f0e1")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import math

torch.manual_seed(0)

# Determine parameters for compute_viterbi
# P: [0,1]^{B x m x 2^(o+1)}
# We need to choose B, m, and o such that P has shape [B x m x 2^(o+1)]

B = 2  # batch size
m = 3  # sequence length
o = 2  # order parameter, so 2^(o+1) = 2^3 = 8

# P should have shape [B x m x 2^(o+1)] = [2 x 3 x 8]
P = torch.rand(B, m, 2**(o+1))
P = torch.clamp(P, 0, 1)  # Ensure values in [0,1]

# quads can be None or a precomputed tensor
# When None, the function will compute it via precompute_quads(o)
quads = None

ARGS = {
    "P": P,
    "quads": quads,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
cosine_distance_torch Ran
ran; our contract draft was wrong, not the code
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("6fa3b3b0bfe31f58")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 3) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create two small 2D tensors with distinct dimensions
# x1: batch size 2, embedding dim 4
x1 = torch.randn(2, 4)

# x2: batch size 3, embedding dim 4 (distinct batch size for probing)
x2 = torch.randn(3, 4)

# eps parameter for numerical stability
eps = 1e-08

ARGS = {
    'x1': x1,
    'x2': x2,
    'eps': eps
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 3) — shape (2, 3) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
count_correct_matches Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("9272637653b59862")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned int, ndim 0, dtype family int, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create small binary encodings tensors
# encodings1: shape (2, 4) - 2 samples, 4-dimensional binary codes
encodings1 = torch.randint(0, 2, (2, 4), dtype=torch.float32)

# encodings2: shape (3, 4) - 3 samples, 4-dimensional binary codes
encodings2 = torch.randint(0, 2, (3, 4), dtype=torch.float32)

ARGS = {
    'encodings1': encodings1,
    'encodings2': encodings2,
    'distance_metric': 'hamming',
    'chunk_size': 100,
    'binary': True,
    'num_retrieve': 2
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type int, ndim 0, dtype family int, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
encode_related_articles Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("cfee1418a588fa69")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 3, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Create a simple encoder function that processes tensors
def simple_encoder(x):
    # Just returns a transformed version of the input
    return x * 2.0

# Create a data loader as a list of (X, Y) tuples
# Each batch has 2 samples, with feature dimension 4
batch_X_1 = torch.randn(2, 4)
batch_Y_1 = torch.randn(2, 4)
batch_X_2 = torch.randn(2, 4)
batch_Y_2 = torch.randn(2, 4)

eval_loader = [
    (batch_X_1, batch_Y_1),
    (batch_X_2, batch_Y_2),
]

device = "cpu"

encode_discrete = simple_encoder

ARGS = {
    "eval_loader": eval_loader,
    "device": device,
    "encode_discrete": encode_discrete,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 3, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
estimate_hZ Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("ac69d5b0b6d23fd2")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import math

torch.manual_seed(0)

# Construct a valid P tensor
# P shape should be [B x m x 2^(o+1)]
# Let's use B=2, m=4, o=3
# So P shape is [2, 4, 2^(3+1)] = [2, 4, 16]
B = 2
m = 4
o = 3
P = torch.rand(B, m, 2**(o+1))
# Ensure P values are in [0,1] - already satisfied by torch.rand
# Add small epsilon to avoid log(0)
P = P + 1e-8
P = P / P.sum(dim=2, keepdim=True)  # normalize to valid probabilities

# I can be None (will be computed by the function) or precomputed
# Let's precompute it to have a valid tensor argument
I = torch.randint(0, 2**(o+1), (B, m, 2**m))

ARGS = {
    "P": P,
    "I": I
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
estimate_hZ_X Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("0a877cffaa839917")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import math

torch.manual_seed(0)

# Parameters for estimate_hZ_X
B = 2  # batch size
m = 3  # sequence length
o = 1  # log2 of half the P dimension
r = 2  # log2 of half the Q_ dimension

# P: [0,1]^{B x m x 2^(o+1)}
# With o=1: 2^(o+1) = 2^2 = 4
P = torch.rand(B, m, pow(2, o + 1))

# Q_: R^{B x m x 2^(r+1)}
# With r=2: 2^(r+1) = 2^3 = 8
Q_ = torch.randn(B, m, pow(2, r + 1))

# quads: precomputed index quadruples for o=1
def precompute_quads(o):
    import itertools
    zs = [''.join(map(str, bits)) for bits in itertools.product([0, 1], repeat=o)]
    quads = [(int(z[1:] + '0', 2), int(z + '0', 2), int(z[1:] + '1', 2), int(z + '1', 2)) for z in zs]
    return torch.LongTensor(quads)

quads = precompute_quads(o)

# buffers: precomputed buffers for i=1...r-o
def precompute_buffers(B, o, r, device):
    buffers = []
    for i in range(r - o):
        buffer = torch.zeros(B, 1, pow(2, o + 1 + i)).to(device)
        buffer[:, 0, 0].fill_(1)
        buffers.append(buffer)
    return buffers

buffers = precompute_buffers(B, o, r, torch.device('cpu'))

ARGS = {
    'P': P,
    'Q_': Q_,
    'quads': quads,
    'buffers': buffers
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_init_function Ran
ran; our contract draft was wrong, not the code
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("e57d5ab85f745b35")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned function. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# The function get_init_function takes init_value as a parameter
# It returns an init_function that initializes weights and biases of a module
# The init_function is typically used with model.apply()

# Create a simple model to test with
class SimpleModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear1 = nn.Linear(4, 3)
        self.linear2 = nn.Linear(3, 2)
    
    def forward(self, x):
        return self.linear2(self.linear1(x))

# Test the init_value parameter
# init_value should be a positive float for the initialization to work
init_value = 0.5

ARGS = {
    'init_value': init_value
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type function — a function output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a function output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
precompute_I Ran
ran, honoured the contract we drafted
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("4fc17a091271ccf2")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family int, shape (3, 8) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import itertools

torch.manual_seed(0)

# The function precompute_I takes two parameters: m and o
# m: an integer representing the dimension of binary vectors
# o: an integer representing the history/context length, where o <= m

# From the function body:
# - m is used in pow(2, m) to determine tensor dimensions
# - o is used as an offset and must satisfy o <= m
# - The function creates tensors of shape (o+m, 2^m)

# Small values to keep tensors tiny
m = 3  # dimension of binary vectors, results in 2^3 = 8 columns
o = 2  # history length, must be <= m

ARGS = {
    "m": m,
    "o": o,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family int, shape (3, 8) — shape (3, 8) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
precompute_buffers Ran
ran; our contract draft was wrong, not the code
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("b784c979d92e7d01")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Analyze the function:
# - B: batch size (int)
# - o: offset parameter (int)
# - r: range parameter (int)
# - device: torch device
# The function creates r-o buffers, each of shape (B, 1, 2^(o+1+i))
# It fills the first element of each buffer with 1

ARGS = {
    "B": 2,
    "o": 1,
    "r": 4,
    "device": torch.device("cpu")
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list — a list output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
precompute_quads Ran
ran, honoured the contract we drafted
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("4a6768de473fef0d")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family int, shape (8, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import itertools

torch.manual_seed(0)

ARGS = {
    "o": 3
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family int, shape (8, 4) — shape (8, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
AMMI Not yet run
unverified (harvested, no recorded run)
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("833b7b44cb181c6d")
How it ran

no invocation recorded.

Data Not yet run
unverified (harvested, no recorded run)
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("832329ad3f5db304")
How it ran

no invocation recorded.

EntropyHelper Not yet run
unverified (harvested, no recorded run)
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("1994d2ff3d492940")
How it ran

no invocation recorded.

LabeledDocuments Not yet run
unverified (harvested, no recorded run)
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("d784281de8217a41")
How it ran

no invocation recorded.

Model Not yet run
unverified (harvested, no recorded run)
karlstratos/ammi/ammi.py
code served (permissive licence) · get_code("277398c5b8f05b33")
How it ran

no invocation recorded.

Also present in other papers' code

Of 27 samples lifted from this paper's linked repositories, 27 carry a function identity and 0 do not. Of the 27 distinct functions with an identity, 0 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 0 later) and 27 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We propose learning discrete structured representations from unlabeled data by maximizing the mutual information between a structured latent variable and a target variable. Calculating mutual information is intractable in this setting. Our key technical contribution is an adversarial objective that can be used to tractably estimate mutual information assuming only the feasibility of cross entropy calculation. We develop a concrete realization of this general formulation with Markov distributions over binary encodings. We report critical and unexpected findings on practical aspects of the objective such as the choice of variational priors. We apply our model on document hashing and show that it outperforms current best baselines based on discrete and vector quantized variational autoencoders. It also yields highly compressed interpretable representations.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2004.03991")
get_code_for_paper("2004.03991")
have("2004.03991")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2004.03991.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2004.03991.svg)](https://syntology.ai/paper/2004.03991)

Connect an agent — have() is free.