SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2607.12166 · 2026

From Geometric Recovery to Causal Validation: A Reproducible Audit of Sparse Autoencoder Features, from Superposition Geometry to Causal Inertness

Mohamed Bal

arXiv · PDF · Open in the Atlas

Code that ran

14 of 17 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (3 of them constructed an object rather than computing a result; 2 honoured the contract we drafted, 0 violated it and 3 have no contract verdict; 9 failures of our instrument; 3 no recorded run), measured 2026-11-22; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2607.12166.json · badge: /api/ran/2607.12166.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
mohamed-bal/sae-causal-audit canonical (classifier label)
classifier basis: census
5 ran · 1 no recorded run
mohamed-bal/superposition-to-monosemanticity community
Papers with Code archive
9 ran · 2 no recorded run
FunctionStatusWhere it lives
TopKSAEConfig Ran
ran on a synthesized input · constructed an object; no result was computed
mohamed-bal/superposition-to-monosemanticity/src/part3_causal_validation.py
code served (permissive licence) · get_code("89f335d65e7982f3")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, TopKSAEConfig: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
from dataclasses import dataclass

torch.manual_seed(0)

ARGS = {
    "n_hidden": 4,
    "dictionary_size": 8,
    "k": 3,
    "seed": 42,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type TopKSAEConfig — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a TopKSAEConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
TopKSparseAutoencoder Ran
ran on a synthesized input · constructed an object; no result was computed
mohamed-bal/superposition-to-monosemanticity/src/part3_causal_validation.py
code served (permissive licence) · get_code("c238475dfc3766c1")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, TopKSparseAutoencoder: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from dataclasses import dataclass

torch.manual_seed(0)

@dataclass(frozen=True)
class TopKSAEConfig:
    """Configuration for a TopK sparse autoencoder.

    Attributes:
        n_hidden: Dimensionality of the activations being explained.
        dictionary_size: Number of learned dictionary features.
        k: Exact number of active (nonzero) code entries per sample —
           directly sets L0, unlike the indirect L1_coeff -> L0 relationship
           in the L1 SAE.
        seed: RNG seed for reproducible initialization.
    """
    n_hidden: int
    dictionary_size: int
    k: int
    seed: int = 0

    def __post_init__(self) -> None:
        if self.n_hidden < 1 or self.dictionary_size < 1:
            raise ValueError('n_hidden and dictionary_size must both be >= 1.')
        if not 1 <= self.k <= self.dictionary_size:
            raise ValueError(f'k must be in [1, dictionary_size] (dictionary_size={self.dictionary_size}), got k={self.k}.')

# Create a minimal config
config = TopKSAEConfig(
    n_hidden=4,
    dictionary_size=8,
    k=2,
    seed=0
)

ARGS = {
    'config': config
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type TopKSparseAutoencoder — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a TopKSparseAutoencoder output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ToyReluOutputModel Ran
ran on a synthesized input · constructed an object; no result was computed
mohamed-bal/superposition-to-monosemanticity/src/part3_causal_validation.py
code served (permissive licence) · get_code("3c8dcf901760d6cc")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ToyReluOutputModel: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from dataclasses import dataclass

torch.manual_seed(0)

@dataclass(frozen=True)
class ToyModelConfig:
    """Configuration for a single toy superposition model run.

    Attributes:
        n_features: Number of ground-truth synthetic features (input/output dim).
        n_hidden: Bottleneck dimension the model must compress features into.
                  Superposition is only possible/interesting when n_hidden < n_features.
        sparsity: Probability that any given feature is INACTIVE (zeroed) in a
                  sample. sparsity=0.0 -> dense inputs; sparsity=0.99 -> ~1% of
                  features active per sample, the regime where superposition emerges.
        importance_decay: Per-feature importance I_i = importance_decay ** i.
                  Values <1 mean earlier-indexed features are more important,
                  which breaks symmetry so the model has a non-arbitrary
                  incentive for *which* features to represent accurately when
                  it cannot represent all of them equally well.
        seed: RNG seed for full reproducibility of data generation and init.

    Raises:
        ValueError: for any non-physical configuration (raised eagerly in
        __post_init__ rather than surfacing as a confusing shape/NaN error
        deep inside training).
    """
    n_features: int
    n_hidden: int
    sparsity: float
    importance_decay: float = 0.9
    seed: int = 0

    def __post_init__(self) -> None:
        if self.n_features < 1 or self.n_hidden < 1:
            raise ValueError('n_features and n_hidden must both be >= 1.')
        if not 0.0 <= self.sparsity < 1.0:
            raise ValueError(f'sparsity must be in [0, 1), got {self.sparsity}.')
        if not 0.0 < self.importance_decay <= 1.0:
            raise ValueError('importance_decay must be in (0, 1].')

# Create a valid config with n_hidden < n_features for superposition to be interesting
config = ToyModelConfig(
    n_features=4,
    n_hidden=2,
    sparsity=0.5,
    importance_decay=0.9,
    seed=0
)

ARGS = {
    "config": config
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ToyReluOutputModel — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ToyReluOutputModel output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ablation_effect Ran
ran; our contract draft was wrong, not the code
mohamed-bal/superposition-to-monosemanticity/src/part3_causal_validation.py
code served (permissive licence) · get_code("2934a74b1224c7dc")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from dataclasses import dataclass

torch.manual_seed(0)

@dataclass(frozen=True)
class ToyModelConfig:
    n_features: int
    n_hidden: int
    sparsity: float
    importance_decay: float = 0.9
    seed: int = 0

    def __post_init__(self) -> None:
        if self.n_features < 1 or self.n_hidden < 1:
            raise ValueError('n_features and n_hidden must both be >= 1.')
        if not 0.0 <= self.sparsity < 1.0:
            raise ValueError(f'sparsity must be in [0, 1), got {self.sparsity}.')
        if not 0.0 < self.importance_decay <= 1.0:
            raise ValueError('importance_decay must be in (0, 1].')

class ToyReluOutputModel(nn.Module):
    def __init__(self, config: ToyModelConfig):
        super().__init__()
        self.config = config
        generator = torch.Generator().manual_seed(config.seed)
        W_init = torch.randn(config.n_hidden, config.n_features, generator=generator) * (1.0 / config.n_features ** 0.5)
        self.W = nn.Parameter(W_init)
        self.b = nn.Parameter(torch.zeros(config.n_features))

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        if x.shape[-1] != self.config.n_features:
            raise ValueError(f'Expected input with last dim {self.config.n_features}, got {x.shape[-1]}.')
        hidden = x @ self.W.T
        out = hidden @ self.W
        return torch.relu(out + self.b)

    def importance_weights(self) -> torch.Tensor:
        i = torch.arange(self.config.n_features, dtype=torch.float32)
        return self.config.importance_decay ** i

    def interference_matrix(self) -> torch.Tensor:
        with torch.no_grad():
            return self.W.T @ self.W

def isolated_feature_batch(n_features: int, feature_idx: int, n_samples: int, generator: torch.Generator) -> torch.Tensor:
    x = torch.zeros(n_samples, n_features)
    x[:, feature_idx] = torch.rand(n_samples, generator=generator)
    return x

# Create a toy model
config = ToyModelConfig(n_features=4, n_hidden=3, sparsity=0.5, seed=0)
toy_model = ToyReluOutputModel(config)

# Create simple SAE encode/decode functions
# sae_encode_fn: takes hidden (batch, n_hidden) -> latent (batch, n_atoms)
# sae_decode_fn: takes latent (batch, n_atoms) -> reconstructed hidden (batch, n_hidden)
n_atoms = 5

def sae_encode_fn(h: torch.Tensor) -> torch.Tensor:
    # Simple linear projection to latent space
    batch_size = h.shape[0]
    return torch.randn(batch_size, n_atoms)

def sae_decode_fn(f: torch.Tensor) -> torch.Tensor:
    # Simple linear projection back to hidden space
    batch_size = f.shape[0]
    return torch.randn(batch_size, config.n_hidden)

ARGS = {
    'toy_model': toy_model,
    'sae_encode_fn': sae_encode_fn,
    'sae_decode_fn': sae_decode_fn,
    'feature_idx': 0,
    'atom_idx': 1,
    'n_samples': 2,
    'seed': 0,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
best_match_cosine_similarity Ran
ran; our contract draft was wrong, not the code
mohamed-bal/superposition-to-monosemanticity/src/part3_causal_validation.py
code served (permissive licence) · get_code("6746fb8078ac1515")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 1, dtype family float, shape (2,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create ground truth directions: (n_features=2, n_hidden=4)
# These should be unit-normalized
ground_truth = torch.randn(2, 4)
ground_truth = ground_truth / ground_truth.norm(dim=1, keepdim=True)

# Create dictionary atoms: (dictionary_size=3, n_hidden=4)
# These don't need to be pre-normalized according to the docstring
dictionary = torch.randn(3, 4)

ARGS = {
    "ground_truth_dirs": ground_truth,
    "dictionary_atoms": dictionary,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 1, dtype family float, shape (2,) — shape (2,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
best_match_indices_and_signs Ran
ran; our contract draft was wrong, not the code
mohamed-bal/superposition-to-monosemanticity/src/part3_causal_validation.py
code served (permissive licence) · get_code("0e05eeba26edf7b5")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create ground_truth_dirs: shape (batch_size, feature_dim)
# These represent the true directions of features
ground_truth_dirs = torch.randn(2, 4)

# Create dictionary_atoms: shape (num_atoms, feature_dim)
# These represent the learned dictionary atoms
dictionary_atoms = torch.randn(3, 4)

ARGS = {
    "ground_truth_dirs": ground_truth_dirs,
    "dictionary_atoms": dictionary_atoms,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 2, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
isolated_feature_batch Ran
ran, honoured the contract we drafted
mohamed-bal/superposition-to-monosemanticity/src/part3_causal_validation.py
code served (permissive licence) · get_code("048ad71b27e5569d")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

ARGS = {
    "n_features": 4,
    "feature_idx": 1,
    "n_samples": 2,
    "generator": torch.Generator()
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
load_json Ran
ran; our contract draft was wrong, not the code
mohamed-bal/sae-causal-audit/src/sae_causal_audit/report.py
code served (permissive licence) · get_code("fe76107a48f0807c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import json
import tempfile
from pathlib import Path

# Create a temporary JSON file with test data including inf values
temp_dir = tempfile.TemporaryDirectory()
temp_path = Path(temp_dir.name) / "test_report.json"

# Create a test JSON file with various data types including inf strings
test_data = {
    "metrics": {
        "loss": 0.5,
        "accuracy": 0.95,
        "max_value": "inf",
        "min_value": "-inf"
    },
    "nested": {
        "list": [1, 2, "inf", "-inf"],
        "deep": {
            "value": "inf"
        }
    },
    "simple": 42
}

# Write the test data to the temp file
temp_path.write_text(json.dumps(test_data), encoding='utf-8')

ARGS = {
    "path": temp_path
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
match_features_to_atoms Ran
ran; our contract draft was wrong, not the code
mohamed-bal/sae-causal-audit/src/sae_causal_audit/matching.py
code served (permissive licence) · get_code("c9e4ddddf99de7b8")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create small tensors with distinct dimensions to test the function
# true_directions: (n_features, d_in) - 2 features, 4 dimensions
true_directions = torch.randn(2, 4, dtype=torch.float32)

# dictionary: (d_sae, d_in) - 3 atoms, 4 dimensions
dictionary = torch.randn(3, 4, dtype=torch.float32)

ARGS = {
    "true_directions": true_directions,
    "dictionary": dictionary,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
steering_effect Ran
ran; our contract draft was wrong, not the code
mohamed-bal/superposition-to-monosemanticity/src/part3_causal_validation.py
code served (permissive licence) · get_code("9c612b80787dd24d")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dict. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from dataclasses import dataclass

torch.manual_seed(0)

@dataclass(frozen=True)
class ToyModelConfig:
    n_features: int
    n_hidden: int
    sparsity: float
    importance_decay: float = 0.9
    seed: int = 0

    def __post_init__(self) -> None:
        if self.n_features < 1 or self.n_hidden < 1:
            raise ValueError('n_features and n_hidden must both be >= 1.')
        if not 0.0 <= self.sparsity < 1.0:
            raise ValueError(f'sparsity must be in [0, 1), got {self.sparsity}.')
        if not 0.0 < self.importance_decay <= 1.0:
            raise ValueError('importance_decay must be in (0, 1].')

class ToyReluOutputModel(nn.Module):
    def __init__(self, config: ToyModelConfig):
        super().__init__()
        self.config = config
        generator = torch.Generator().manual_seed(config.seed)
        W_init = torch.randn(config.n_hidden, config.n_features, generator=generator) * (1.0 / config.n_features ** 0.5)
        self.W = nn.Parameter(W_init)
        self.b = nn.Parameter(torch.zeros(config.n_features))

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        if x.shape[-1] != self.config.n_features:
            raise ValueError(f'Expected input with last dim {self.config.n_features}, got {x.shape[-1]}.')
        hidden = x @ self.W.T
        out = hidden @ self.W
        return torch.relu(out + self.b)

    def importance_weights(self) -> torch.Tensor:
        i = torch.arange(self.config.n_features, dtype=torch.float32)
        return self.config.importance_decay ** i

    def interference_matrix(self) -> torch.Tensor:
        with torch.no_grad():
            return self.W.T @ self.W

class SimpleSAE(nn.Module):
    """Minimal SAE-like module with encode and decode methods."""
    def __init__(self, hidden_dim: int, latent_dim: int):
        super().__init__()
        self.encoder = nn.Linear(hidden_dim, latent_dim)
        self.decoder = nn.Linear(latent_dim, hidden_dim)
    
    def encode(self, x: torch.Tensor) -> torch.Tensor:
        return self.encoder(x)
    
    def decode(self, z: torch.Tensor) -> torch.Tensor:
        return self.decoder(z)

config = ToyModelConfig(n_features=4, n_hidden=3, sparsity=0.5, seed=0)
toy_model = ToyReluOutputModel(config)

sae = SimpleSAE(hidden_dim=3, latent_dim=5)

def sae_encode_fn(h: torch.Tensor) -> torch.Tensor:
    return sae.encode(h)

def sae_decode_fn(f: torch.Tensor) -> torch.Tensor:
    return sae.decode(f)

ARGS = {
    "toy_model": toy_model,
    "sae_encode_fn": sae_encode_fn,
    "sae_decode_fn": sae_decode_fn,
    "feature_idx": 0,
    "atom_idx": 2,
    "steering_value": 0.5,
    "n_samples": 2,
    "seed": 0,
    "sign": 1.0,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type dict — a dict output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dict output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
train_topk_sae Ran
ran; our contract draft was wrong, not the code
mohamed-bal/superposition-to-monosemanticity/src/part3_causal_validation.py
code served (permissive licence) · get_code("99f04d71b917b33b")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from dataclasses import dataclass

torch.manual_seed(0)

@dataclass(frozen=True)
class TopKSAEConfig:
    """Configuration for a TopK sparse autoencoder."""
    n_hidden: int
    dictionary_size: int
    k: int
    seed: int = 0

    def __post_init__(self) -> None:
        if self.n_hidden < 1 or self.dictionary_size < 1:
            raise ValueError('n_hidden and dictionary_size must both be >= 1.')
        if not 1 <= self.k <= self.dictionary_size:
            raise ValueError(f'k must be in [1, dictionary_size] (dictionary_size={self.dictionary_size}), got k={self.k}.')

# Create a minimal config for the SAE
config = TopKSAEConfig(
    n_hidden=4,
    dictionary_size=8,
    k=2,
    seed=0
)

# Create a small tensor of activations: (batch_size, n_hidden)
# Using 16 samples of dimension 4
activations = torch.randn(16, 4)

# Arguments for train_topk_sae
ARGS = {
    'config': config,
    'activations': activations,
    'n_steps': 10,
    'batch_size': 4,
    'lr': 0.001,
    'dead_latent_window': 3,
    'aux_k_multiplier': 2,
    'aux_loss_coeff': 1.0 / 32,
    'log_every': 5
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
train_topk_sae Ran
ran; our contract draft was wrong, not the code
mohamed-bal/sae-causal-audit/src/sae_causal_audit/toy.py
code served (permissive licence) · get_code("8abb4bea3d094422")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned TopKSAE. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn

torch.manual_seed(0)

# Define ToyConfig
class ToyConfig:
    def __init__(self, n_features=4, n_hidden=3, sparsity=0.95, importance_decay=0.9, seed=0):
        self.n_features = n_features
        self.n_hidden = n_hidden
        self.sparsity = sparsity
        self.importance_decay = importance_decay
        self.seed = seed

# Define ToyModel
class ToyModel(nn.Module):
    def __init__(self, cfg):
        super().__init__()
        self.cfg = cfg
        gen = torch.Generator().manual_seed(cfg.seed)
        self.W = nn.Parameter(torch.randn(cfg.n_hidden, cfg.n_features, generator=gen) * 0.1)
        self.b = nn.Parameter(torch.zeros(cfg.n_features))

    def sample_batch(self, n, generator):
        x = torch.rand(n, self.cfg.n_features, generator=generator)
        mask = torch.rand(n, self.cfg.n_features, generator=generator) >= self.cfg.sparsity
        return x * mask

    def hidden(self, x):
        return x @ self.W.T

    def output(self, h):
        return torch.relu(h @ self.W + self.b)

    def importance(self):
        i = torch.arange(self.cfg.n_features, dtype=torch.float32)
        return self.cfg.importance_decay ** i

# Create a ToyModel instance
config = ToyConfig(n_features=4, n_hidden=3, sparsity=0.95, importance_decay=0.9, seed=0)
toy_model = ToyModel(config)

# Define parameters for train_topk_sae
ARGS = {
    "model": toy_model,
    "d_sae": 5,
    "k": 2,
    "steps": 2,
    "batch": 2,
    "seed": 0
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type TopKSAE — a TopKSAE output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a TopKSAE output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
train_toy_model Ran
ran; our contract draft was wrong, not the code
mohamed-bal/sae-causal-audit/src/sae_causal_audit/toy.py
code served (permissive licence) · get_code("df5d7672c5a4c9ea")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ToyModel. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn

torch.manual_seed(0)

from dataclasses import dataclass

@dataclass(frozen=True, slots=True)
class ToyConfig:
    n_features: int = 4
    n_hidden: int = 2
    sparsity: float = 0.95
    importance_decay: float = 0.9
    seed: int = 0

    def __post_init__(self) -> None:
        if self.n_hidden <= 0 or self.n_features <= 0:
            raise ValueError('dimensions must be positive')
        if not 0.0 <= self.sparsity < 1.0:
            raise ValueError('sparsity must be in [0, 1)')

cfg = ToyConfig(
    n_features=4,
    n_hidden=2,
    sparsity=0.95,
    importance_decay=0.9,
    seed=0
)

ARGS = {
    "cfg": cfg,
    "steps": 2,
    "batch": 2
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ToyModel — a ToyModel output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a ToyModel output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
true_directions Ran
ran, honoured the contract we drafted
mohamed-bal/sae-causal-audit/src/sae_causal_audit/toy.py
code served (permissive licence) · get_code("7ef3e00364872350")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (4, 3) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn
from dataclasses import dataclass

torch.manual_seed(0)

@dataclass(frozen=True, slots=True)
class ToyConfig:
    n_features: int = 32
    n_hidden: int = 8
    sparsity: float = 0.95
    importance_decay: float = 0.9
    seed: int = 0

    def __post_init__(self) -> None:
        if self.n_hidden <= 0 or self.n_features <= 0:
            raise ValueError('dimensions must be positive')
        if not 0.0 <= self.sparsity < 1.0:
            raise ValueError('sparsity must be in [0, 1)')

class ToyModel(nn.Module):
    """``h = W x``; ``x_hat = ReLU(Wᵀ h + b)`` — the bottleneck model."""

    def __init__(self, cfg: ToyConfig) -> None:
        super().__init__()
        self.cfg = cfg
        gen = torch.Generator().manual_seed(cfg.seed)
        self.W = nn.Parameter(torch.randn(cfg.n_hidden, cfg.n_features, generator=gen) * 0.1)
        self.b = nn.Parameter(torch.zeros(cfg.n_features))

    def sample_batch(self, n: int, generator: torch.Generator) -> torch.Tensor:
        x = torch.rand(n, self.cfg.n_features, generator=generator)
        mask = torch.rand(n, self.cfg.n_features, generator=generator) >= self.cfg.sparsity
        return x * mask

    def hidden(self, x: torch.Tensor) -> torch.Tensor:
        return x @ self.W.T

    def output(self, h: torch.Tensor) -> torch.Tensor:
        return torch.relu(h @ self.W + self.b)

    def importance(self) -> torch.Tensor:
        i = torch.arange(self.cfg.n_features, dtype=torch.float32)
        return self.cfg.importance_decay ** i

def true_directions(model: ToyModel) -> torch.Tensor:
    """Ground-truth feature directions in activation space: columns of W."""
    return model.W.detach().T.clone()

cfg = ToyConfig(n_features=4, n_hidden=3, sparsity=0.5, importance_decay=0.9, seed=0)
model = ToyModel(cfg)

ARGS = {
    "model": model,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (4, 3) — shape (4, 3) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ToyModelConfig Not yet run
unverified (harvested, no recorded run)
mohamed-bal/superposition-to-monosemanticity/src/part3_causal_validation.py
code served (permissive licence) · get_code("4b36c0146dcd4f00")
How it ran

no invocation recorded.

bootstrap_ci Not yet run
unverified (harvested, no recorded run)
mohamed-bal/sae-causal-audit/src/sae_causal_audit/stats.py
code served (permissive licence) · get_code("cd4f9aebcb38b4ea")
How it ran

no invocation recorded.

run_causal_study_for_sae Not yet run
unverified (harvested, no recorded run)
mohamed-bal/superposition-to-monosemanticity/src/part3_causal_validation.py
code served (permissive licence) · get_code("b14e1bd32da0dbfc")
How it ran

no invocation recorded.

Also present in other papers' code

Of 17 samples lifted from this paper's linked repositories, 17 carry a function identity and 0 do not. Of the 17 distinct functions with an identity, 0 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 0 later) and 17 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Sparse autoencoders (SAEs) are the de facto standard for decomposing superposed neural representations into interpretable features, and the field's evaluation practice relies predominantly on correlational recovery metrics-cosine similarity between ground-truth (or probed) directions and learned decoder atoms. We show, in a fully controlled setting where ground truth is known, that this practice conflates two distinct empirical claims: decoder-geometry alignment and encoder-activation behavior. We first reproduce, from scratch and with multi-seed statistics, the superposition phase diagram of Elhage et al. (2022), identifying along the way a convergence artifact at high sparsity and a previously under-described diffuse sharing regime at extreme overcompleteness. We then reproduce the TopK-versus-L 1 SAE comparison of Gao et al. (2024), including direct quantitative evidence of L 1 shrinkage (activation-magnitude refinement recovers 91.2% of the L 1 SAE's reconstruction gap while shifting magnitudes +22.5%, versus -0.05% for TopK). Our central result is causal: subjecting every correlationally recovered feature to ablation and steering interventions, we find that up to 77% of features passing a standard recovery bar (cosine ≥ 0.90) in a degraded SAE-and 9% in a well-trained one-are causally inert: the matched atom never fires when the feature is present, including matches at cosine ≈ 1.000. We package the methodology as sae-causal-audit, a model-agnostic, protocol-typed instrument with a hash-verified deterministic reproduction pipeline, and re-auditing the setting under it refines the finding twice: causal inertness decomposes by cause into structural inertness (traceable to antipodal-pair superposition geometry and present in good SAEs) and competitive inertness (a TopK-selection pathology of degraded SAEs), and by direction into read-inertness and write-inertness, which five antipodal pairs dissociate completely-features that are unmonitorable yet steerable through the same atom, with steering specificities of 143-310 attached to ablation effects of exactly zero. We additionally document why byte-exact cross-platform reproducibility is unavailable by construction for this class of workload, and propose reporting reproducibility as a stack of claims with explicit scopes. Applying the instrument to a published production SAE (GPT-2-small, 83 concepts, one hook layer) reproduces the qualitative pattern at small scale (14% causally inert among recovered features) and surfaces an unanticipated real-model signal: a handful of decoder atoms recur as the nearest correlational match for dozens of semantically unrelated concepts, replicated across three independently constructed concept batches-realmodel evidence for the same dictionary under-splitting the toy antipodal-pair mechanism was built to make legible.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2607.12166")
get_code_for_paper("2607.12166")
have("2607.12166")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2607.12166.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2607.12166.svg)](https://syntology.ai/paper/2607.12166)

Connect an agent — have() is free.