SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2606.01495 · 2026

CART: Context-Anchored Recurrent Transformer * A Parameter-Efficient Architecture with Learned Stability

Chad Capps

arXiv · PDF · Open in the Atlas

Code that ran

7 of 16 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 0 violated it and 6 have no contract verdict; 1 failure of our instrument; 9 no recorded run), measured 2026-11-22; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2606.01495.json · badge: /api/ran/2606.01495.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
ccapps42/CART community
Papers with Code archive
7 ran · 9 no recorded run
FunctionStatusWhere it lives
HyperConnection Ran
ran on a synthesized input
ccapps42/CART/model/cart.py
code served (permissive licence) · get_code("e70ccca8c4433ab8")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from dataclasses import dataclass
import math

@dataclass
class CARTConfig:
    d_model: int = 512
    n_loops: int = 6
    n_prelude: int = 4
    n_coda: int = 1
    vocab_size: int = 32000
    max_seq_len: int = 1024
    d_head: int = 64
    mla_compression_ratio: int = 4
    n_hyper: int = 3
    lti_init_value: float = 0.9
    lie_dim: int = 32
    rms_norm_eps: float = 1e-06
    tie_embeddings: bool = True
    rope_base: float = 10000.0
    dropout: float = 0.0
    unfreeze_kv: bool = False
    unshare_core: bool = False
    self_attn_core: bool = False
    disable_hyper: bool = False
    disable_lti: bool = False
    disable_lie: bool = False

    @property
    def n_heads(self) -> int:
        assert self.d_model % self.d_head == 0, f'd_model {self.d_model} must be divisible by d_head {self.d_head}'
        return self.d_model // self.d_head

    @property
    def d_kv_latent(self) -> int:
        return self.d_model // self.mla_compression_ratio

    @property
    def ffn_intermediate(self) -> int:
        nominal = int(8 / 3 * self.d_model)
        return math.ceil(nominal / 256) * 256

    def validate(self):
        assert self.d_model % 64 == 0, 'd_model must be divisible by 64'
        assert self.n_loops >= 1, 'n_loops must be at least 1 (R=1 is the no-recurrence diagnostic)'
        assert self.n_prelude >= 2, 'n_prelude must be at least 2'
        assert self.n_coda == 1, 'n_coda must be 1 (fixed)'
        assert not (self.self_attn_core and self.unfreeze_kv), 'self_attn_core=True makes unfreeze_kv meaningless (no K, V cache exists)'

class HyperConnection(nn.Module):
    """
    Hyper-connection mechanism at loop boundaries.
    Maintains a ring buffer of the last n_hyper hidden states.
    Combines them with learned scalar weights initialized to residual baseline:
        weights = [1.0, 0.0, 0.0, ...]  (w0 = most recent, w_{n-1} = oldest)

    At loop iteration r:
        - Buffer holds states [h_{r-1}, h_{r-2}, ..., h_{r-n}]
        - Zero-padded for the first n-1 iterations
        - h_input = sum(w_i * buffer[i])

    With residual initialization, this behaves identically to standard
    residual connections until the training data supports learning non-zero
    weights for older states. The learned weight vector is a paper result.
    """

    def __init__(self, config: CARTConfig):
        super().__init__()
        n = config.n_hyper
        init = torch.zeros(n)
        init[0] = 1.0
        self.weights = nn.Parameter(init)
        self.n_hyper = n

    def init_buffer(self, h: torch.Tensor) -> list:
        """
        Initialize the ring buffer before the loop starts.
        All slots set to h (the initial hidden state from prelude output).
        With residual init weights [1,0,0] this is safe regardless of
        what fills the non-primary slots.
        """
        return [h.clone() for _ in range(self.n_hyper)]

    def combine(self, buffer: list) -> torch.Tensor:
        """
        buffer[0] = h_{r-1} (most recent)
        buffer[1] = h_{r-2}
        buffer[2] = h_{r-3}
        Returns the weighted combination as h_input for this iteration.
        """
        w = torch.softmax(self.weights, dim=0)
        result = sum((w[i] * buffer[i] for i in range(self.n_hyper)))
        return result

    def update_buffer(self, buffer: list, h_new: torch.Tensor) -> list:
        """Shift buffer and insert new hidden state at position 0."""
        return [h_new.clone()] + buffer[:-1]


config = CARTConfig(
    d_model=64,
    n_hyper=3,
    n_loops=2,
    n_prelude=2,
    n_coda=1,
    vocab_size=128,
    max_seq_len=32,
    d_head=16
)

INIT_ARGS = {
    "config": config
}

batch_size = 2
seq_len = 5
d_model = 64

h_tensor = torch.randn(batch_size, seq_len, d_model)

hyper_conn = HyperConnection(**INIT_ARGS)
buffer = hyper_conn.init_buffer(h_tensor)

ARGS = {
    "buffer": buffer
}

METHOD = "combine"
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 3, dtype family float — no output shape is stated: observe_signature.py observed this class with its constructor sizes canonicalised where it could, which the node does not record, so its shape is not stated as this setup block's
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
LTIInjection Ran
ran on a synthesized input
ccapps42/CART/model/cart.py
code served (permissive licence) · get_code("34d128600564fc2a")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float, shape (2, 3, 64) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from dataclasses import dataclass
import math

@dataclass
class CARTConfig:
    d_model: int = 512
    n_loops: int = 6
    n_prelude: int = 4
    n_coda: int = 1
    vocab_size: int = 32000
    max_seq_len: int = 1024
    d_head: int = 64
    mla_compression_ratio: int = 4
    n_hyper: int = 3
    lti_init_value: float = 0.9
    lie_dim: int = 32
    rms_norm_eps: float = 1e-06
    tie_embeddings: bool = True
    rope_base: float = 10000.0
    dropout: float = 0.0
    unfreeze_kv: bool = False
    unshare_core: bool = False
    self_attn_core: bool = False
    disable_hyper: bool = False
    disable_lti: bool = False
    disable_lie: bool = False

    @property
    def n_heads(self) -> int:
        assert self.d_model % self.d_head == 0, f'd_model {self.d_model} must be divisible by d_head {self.d_head}'
        return self.d_model // self.d_head

    @property
    def d_kv_latent(self) -> int:
        return self.d_model // self.mla_compression_ratio

    @property
    def ffn_intermediate(self) -> int:
        nominal = int(8 / 3 * self.d_model)
        return math.ceil(nominal / 256) * 256

    def validate(self):
        assert self.d_model % 64 == 0, 'd_model must be divisible by 64'
        assert self.n_loops >= 1, 'n_loops must be at least 1 (R=1 is the no-recurrence diagnostic)'
        assert self.n_prelude >= 2, 'n_prelude must be at least 2'
        assert self.n_coda == 1, 'n_coda must be 1 (fixed)'
        assert not (self.self_attn_core and self.unfreeze_kv), 'self_attn_core=True makes unfreeze_kv meaningless (no K, V cache exists)'

class LTIInjection(nn.Module):
    """
    LTI-stable recurrent injection (Parcae, Prairie et al. 2026).

    Replaces the standard residual connection:
        h = h_input + transformer_out
    with:
        h = A * h_input + transformer_out

    A is a learnable diagonal matrix parameterized as sigmoid(a_param),
    which guarantees every diagonal entry is in (0, 1) and therefore
    the spectral radius rho(A) < 1 by construction. This prevents
    residual explosion at high loop counts (R=6, R=8).

    Initialization: a_param = sigmoid_inverse(lti_init_value) so that
    A starts at lti_init_value (default 0.9) — stable but close to
    standard residual behavior (A=1). The model learns to tighten or
    relax A during training.

    The learned A values are a paper result: if A settles near 0.9
    uniformly, the forgetting mechanism is inert. If A is smaller at
    high R than low R, the model has learned to discard stale loop
    states more aggressively when it has more iterations available.
    Log rho(A) periodically during training.
    """

    def __init__(self, config: CARTConfig):
        super().__init__()
        v = config.lti_init_value
        init_val = math.log(v / (1.0 - v))
        self.a_param = nn.Parameter(torch.full((config.d_model,), init_val))

    def forward(self, h_input: torch.Tensor, transformer_out: torch.Tensor) -> torch.Tensor:
        A = torch.sigmoid(self.a_param)
        return A * h_input + transformer_out

    def spectral_radius(self) -> float:
        """Max diagonal value of A. Log this periodically during training."""
        with torch.no_grad():
            return torch.sigmoid(self.a_param).max().item()


config = CARTConfig(d_model=64, lti_init_value=0.9)

INIT_ARGS = {
    'config': config
}

ARGS = {
    'h_input': torch.randn(2, 3, 64),
    'transformer_out': torch.randn(2, 3, 64)
}

module = LTIInjection(**INIT_ARGS)
output = module(**ARGS)
spectral_rad = module.spectral_radius()

print(f"Output shape: {output.shape}")
print(f"Spectral radius: {spectral_rad}")
print(f"Output sample: {output[0, 0, :4]}")
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 3, dtype family float, shape (2, 3, 64) — shape (2, 3, 64) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
LoopIndexEmbedding Ran
ran on a synthesized input
ccapps42/CART/model/cart.py
code served (permissive licence) · get_code("85330cf0bbe2d9e8")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float, shape (2, 3, 64) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from dataclasses import dataclass
import math

@dataclass
class CARTConfig:
    d_model: int = 512
    n_loops: int = 6
    n_prelude: int = 4
    n_coda: int = 1
    vocab_size: int = 32000
    max_seq_len: int = 1024
    d_head: int = 64
    mla_compression_ratio: int = 4
    n_hyper: int = 3
    lti_init_value: float = 0.9
    lie_dim: int = 32
    rms_norm_eps: float = 1e-06
    tie_embeddings: bool = True
    rope_base: float = 10000.0
    dropout: float = 0.0
    unfreeze_kv: bool = False
    unshare_core: bool = False
    self_attn_core: bool = False
    disable_hyper: bool = False
    disable_lti: bool = False
    disable_lie: bool = False

    @property
    def n_heads(self) -> int:
        assert self.d_model % self.d_head == 0, f'd_model {self.d_model} must be divisible by d_head {self.d_head}'
        return self.d_model // self.d_head

    @property
    def d_kv_latent(self) -> int:
        return self.d_model // self.mla_compression_ratio

    @property
    def ffn_intermediate(self) -> int:
        nominal = int(8 / 3 * self.d_model)
        return math.ceil(nominal / 256) * 256

    def validate(self):
        assert self.d_model % 64 == 0, 'd_model must be divisible by 64'
        assert self.n_loops >= 1, 'n_loops must be at least 1 (R=1 is the no-recurrence diagnostic)'
        assert self.n_prelude >= 2, 'n_prelude must be at least 2'
        assert self.n_coda == 1, 'n_coda must be 1 (fixed)'
        assert not (self.self_attn_core and self.unfreeze_kv), 'self_attn_core=True makes unfreeze_kv meaningless (no K, V cache exists)'

class LoopIndexEmbedding(nn.Module):
    """
    Sinusoidal loop-index embedding (LIE).

    Injects a signal encoding the current loop iteration r into h_input
    before each CoreBlock pass. This gives the shared-weight block
    positional awareness in the loop dimension — it can learn that
    iteration 1 and iteration 6 are different computational contexts
    warranting different behavior.

    Without LIE, the CoreBlock processes every iteration identically
    except for the changing value of h_t. With LIE, the block can
    learn a genuine computational schedule across loop depth (e.g.,
    coarse pattern matching in early loops, fine-grained refinement
    in late loops).

    Implementation:
        - Pre-compute sinusoidal encodings for loop indices 0..max_loops-1
        - At each loop r, look up pe[r] and project to d_model
        - Add the projected signal to h_input (broadcast across [B, T])

    lie_dim = 32 (fixed, not swept). Parameter cost: 32 * d_model.
    At d=768 that is 24,576 parameters — negligible.
    """

    def __init__(self, config: CARTConfig):
        super().__init__()
        self.lie_dim = config.lie_dim
        self.proj = nn.Linear(config.lie_dim, config.d_model, bias=False)
        pe = self._build_sinusoidal(max_loops=16, lie_dim=config.lie_dim)
        self.register_buffer('pe', pe)

    def _build_sinusoidal(self, max_loops: int, lie_dim: int) -> torch.Tensor:
        pe = torch.zeros(max_loops, lie_dim)
        pos = torch.arange(max_loops).unsqueeze(1).float()
        div = torch.exp(torch.arange(0, lie_dim, 2).float() * -(math.log(10000.0) / lie_dim))
        pe[:, 0::2] = torch.sin(pos * div)
        pe[:, 1::2] = torch.cos(pos * div)
        return pe

    def forward(self, h: torch.Tensor, r: int) -> torch.Tensor:
        signal = self.proj(self.pe[r])
        return h + signal


config = CARTConfig(d_model=64, lie_dim=32)

INIT_ARGS = {
    'config': config
}

ARGS = {
    'h': torch.randn(2, 3, 64),
    'r': 2
}

module = LoopIndexEmbedding(**INIT_ARGS)
output = module(**ARGS)
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 3, dtype family float, shape (2, 3, 64) — shape (2, 3, 64) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
MLACrossAttention Ran
ran on a synthesized input
ccapps42/CART/model/cart.py
code served (permissive licence) · get_code("b175bb6a99225d0b")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from dataclasses import dataclass
import math
import torch.nn.functional as F

@dataclass
class CARTConfig:
    d_model: int = 512
    n_loops: int = 6
    n_prelude: int = 4
    n_coda: int = 1
    vocab_size: int = 32000
    max_seq_len: int = 1024
    d_head: int = 64
    mla_compression_ratio: int = 4
    n_hyper: int = 3
    lti_init_value: float = 0.9
    lie_dim: int = 32
    rms_norm_eps: float = 1e-06
    tie_embeddings: bool = True
    rope_base: float = 10000.0
    dropout: float = 0.0
    unfreeze_kv: bool = False
    unshare_core: bool = False
    self_attn_core: bool = False
    disable_hyper: bool = False
    disable_lti: bool = False
    disable_lie: bool = False

    @property
    def n_heads(self) -> int:
        assert self.d_model % self.d_head == 0, f'd_model {self.d_model} must be divisible by d_head {self.d_head}'
        return self.d_model // self.d_head

    @property
    def d_kv_latent(self) -> int:
        return self.d_model // self.mla_compression_ratio

    @property
    def ffn_intermediate(self) -> int:
        nominal = int(8 / 3 * self.d_model)
        return math.ceil(nominal / 256) * 256

    def validate(self):
        assert self.d_model % 64 == 0, 'd_model must be divisible by 64'
        assert self.n_loops >= 1, 'n_loops must be at least 1 (R=1 is the no-recurrence diagnostic)'
        assert self.n_prelude >= 2, 'n_prelude must be at least 2'
        assert self.n_coda == 1, 'n_coda must be 1 (fixed)'
        assert not (self.self_attn_core and self.unfreeze_kv), 'self_attn_core=True makes unfreeze_kv meaningless (no K, V cache exists)'

class MLACrossAttention(nn.Module):
    def __init__(self, config: CARTConfig):
        super().__init__()
        self.n_heads = config.n_heads
        self.d_head = config.d_head
        self.d_model = config.d_model
        self.q_proj = nn.Linear(config.d_model, config.n_heads * config.d_head, bias=False)
        self.o_proj = nn.Linear(config.n_heads * config.d_head, config.d_model, bias=False)
        self.scale = config.d_head ** (-0.5)

    def forward(self, h: torch.Tensor, K: torch.Tensor, V: torch.Tensor) -> torch.Tensor:
        B, T, _ = h.shape
        H, D = (self.n_heads, self.d_head)
        Q = self.q_proj(h).view(B, T, H, D).transpose(1, 2)
        out = F.scaled_dot_product_attention(Q, K, V, is_causal=True)
        out = out.transpose(1, 2).contiguous().view(B, T, H * D)
        return self.o_proj(out)

config = CARTConfig(d_model=8, d_head=4)

INIT_ARGS = {
    'config': config
}

B, T, H, D = 2, 3, config.n_heads, config.d_head

ARGS = {
    'h': torch.randn(B, T, config.d_model),
    'K': torch.randn(B, H, T, D),
    'V': torch.randn(B, H, T, D)
}
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 3, dtype family float — no output shape is stated: observe_signature.py observed this class with its constructor sizes canonicalised where it could, which the node does not record, so its shape is not stated as this setup block's
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
MLAKVProjection Ran
ran on a synthesized input
ccapps42/CART/model/cart.py
code served (permissive licence) · get_code("a21a4d06f59f6f55")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from dataclasses import dataclass
import math

@dataclass
class CARTConfig:
    d_model: int = 512
    n_loops: int = 6
    n_prelude: int = 4
    n_coda: int = 1
    vocab_size: int = 32000
    max_seq_len: int = 1024
    d_head: int = 64
    mla_compression_ratio: int = 4
    n_hyper: int = 3
    lti_init_value: float = 0.9
    lie_dim: int = 32
    rms_norm_eps: float = 1e-06
    tie_embeddings: bool = True
    rope_base: float = 10000.0
    dropout: float = 0.0
    unfreeze_kv: bool = False
    unshare_core: bool = False
    self_attn_core: bool = False
    disable_hyper: bool = False
    disable_lti: bool = False
    disable_lie: bool = False

    @property
    def n_heads(self) -> int:
        assert self.d_model % self.d_head == 0, f'd_model {self.d_model} must be divisible by d_head {self.d_head}'
        return self.d_model // self.d_head

    @property
    def d_kv_latent(self) -> int:
        return self.d_model // self.mla_compression_ratio

    @property
    def ffn_intermediate(self) -> int:
        nominal = int(8 / 3 * self.d_model)
        return math.ceil(nominal / 256) * 256

    def validate(self):
        assert self.d_model % 64 == 0, 'd_model must be divisible by 64'
        assert self.n_loops >= 1, 'n_loops must be at least 1 (R=1 is the no-recurrence diagnostic)'
        assert self.n_prelude >= 2, 'n_prelude must be at least 2'
        assert self.n_coda == 1, 'n_coda must be 1 (fixed)'
        assert not (self.self_attn_core and self.unfreeze_kv), 'self_attn_core=True makes unfreeze_kv meaningless (no K, V cache exists)'

class MLAKVProjection(nn.Module):
    """
    Computes K and V from the prelude output e.
    Called once before the loop begins. Output K, V are passed to
    MLACrossAttention on every loop iteration.
    """

    def __init__(self, config: CARTConfig):
        super().__init__()
        self.n_heads = config.n_heads
        self.d_head = config.d_head
        self.d_kv_latent = config.d_kv_latent
        self.kv_down = nn.Linear(config.d_model, config.d_kv_latent, bias=False)
        self.k_up = nn.Linear(config.d_kv_latent, config.n_heads * config.d_head, bias=False)
        self.v_up = nn.Linear(config.d_kv_latent, config.n_heads * config.d_head, bias=False)

    def forward(self, e: torch.Tensor):
        B, T, _ = e.shape
        H, D = (self.n_heads, self.d_head)
        latent = self.kv_down(e)
        K = self.k_up(latent).view(B, T, H, D).transpose(1, 2)
        V = self.v_up(latent).view(B, T, H, D).transpose(1, 2)
        return (K, V)


config = CARTConfig(
    d_model=64,
    d_head=16,
    mla_compression_ratio=4,
)

INIT_ARGS = {
    'config': config,
}

ARGS = {
    'e': torch.randn(2, 3, 64),
}

if __name__ == '__main__':
    module = MLAKVProjection(**INIT_ARGS)
    output = module(**ARGS)
    print(f"K shape: {output[0].shape}, V shape: {output[1].shape}")
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
RMSNorm Ran
ran on a synthesized input
ccapps42/CART/model/cart.py
code served (permissive licence) · get_code("5e72e01b3a174cc0")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 3, dtype family float, shape (2, 3, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

class RMSNorm(nn.Module):

    def __init__(self, d_model: int, eps: float=1e-06):
        super().__init__()
        self.weight = nn.Parameter(torch.ones(d_model))
        self.eps = eps

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        norm = x.float().pow(2).mean(dim=-1, keepdim=True).add(self.eps).rsqrt()
        return (x.float() * norm).to(x.dtype) * self.weight


INIT_ARGS = {
    "d_model": 4,
    "eps": 1e-06,
}

ARGS = {
    "x": torch.randn(2, 3, 4),
}

if __name__ == "__main__":
    model = RMSNorm(**INIT_ARGS)
    output = model(**ARGS)
    print("Output shape:", output.shape)
    print("Output:", output)
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 3, dtype family float, shape (2, 3, 4) — shape (2, 3, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
rotate_half Ran
ran; our contract draft was wrong, not the code
ccapps42/CART/model/cart.py
code served (permissive licence) · get_code("4da1a1beab34982c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1) and returned Tensor, ndim 2, dtype family float, shape (2, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# The function rotate_half takes a tensor x and:
# 1. Chunks it into 2 parts along the last dimension
# 2. Concatenates [-x2, x1] back together
# So we need a tensor with an even last dimension

x = torch.randn(2, 4)  # batch_size=2, last_dim=4 (even, so it can be chunked into 2)

ARGS = {
    "x": x
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (2, 4) — shape (2, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
CART Not yet run
unverified (harvested, no recorded run)
ccapps42/CART/model/cart.py
code served (permissive licence) · get_code("9da26ca8855b57eb")
How it ran

no invocation recorded.

CARTConfig Not yet run
unverified (harvested, no recorded run)
ccapps42/CART/model/cart.py
code served (permissive licence) · get_code("c74c69070494a0c6")
How it ran

no invocation recorded.

CodaLayer Not yet run
unverified (harvested, no recorded run)
ccapps42/CART/model/cart.py
code served (permissive licence) · get_code("3be28e5f513e072b")
How it ran

no invocation recorded.

CoreBlock Not yet run
unverified (harvested, no recorded run)
ccapps42/CART/model/cart.py
code served (permissive licence) · get_code("a41c55d1f28a5a54")
How it ran

no invocation recorded.

CoreBlockSelfAttn Not yet run
unverified (harvested, no recorded run)
ccapps42/CART/model/cart.py
code served (permissive licence) · get_code("f9264024bf7be927")
How it ran

no invocation recorded.

MLASelfAttention Not yet run
unverified (harvested, no recorded run)
ccapps42/CART/model/cart.py
code served (permissive licence) · get_code("b01f021350a0198e")
How it ran

no invocation recorded.

PreludeLayer Not yet run
unverified (harvested, no recorded run)
ccapps42/CART/model/cart.py
code served (permissive licence) · get_code("e3a8d7c1a3006108")
How it ran

no invocation recorded.

RotaryEmbedding Not yet run
unverified (harvested, no recorded run)
ccapps42/CART/model/cart.py
code served (permissive licence) · get_code("97fbea8bac3c5277")
How it ran

no invocation recorded.

apply_rotary Not yet run
unverified (harvested, no recorded run)
ccapps42/CART/model/cart.py
code served (permissive licence) · get_code("f8acf5fe0065d80e")
How it ran

no invocation recorded.

Also present in other papers' code

Of 16 samples lifted from this paper's linked repositories, 16 carry a function identity and 0 do not. Of the 16 distinct functions with an identity, 0 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 0 later) and 16 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We present CART (Context-Anchored Recurrent Transformer), a parameterefficient language model architecture that reuses a single shared core block R times across depth. Unlike prior looped transformers that recompute key-value representations at every loop iteration, CART computes K and V once from a multi-layer prelude and reuses them throughout the recurrent core via multi-head latent attention (MLA) cross-attention. This design separates context encoding from iterative refinement, reduces per-loop FLOPs, and provides a stable attention anchor for all loop iterations. Stability of the recurrent computation is ensured by a learned Linear Time-Invariant (LTI) gate, which empirically settles on a narrow range of spectral radius values across all model scales (ϱ ∈ [0.79, 0.83] at full training across all 36 configurations). We evaluate CART in two stages on single consumer GPUs (an RTX 3050 for Stage 1, an RTX 3090 for Stage 2). Stage 1 is a 64-configuration hyperparameter screen at 3,000 steps (∼49M tokens) across d ∈ {256, 512, 768, 1024}, R ∈ {2, 4, 6, 8}, P ∈ {2, 3, 4, 6}. Stage 2 trains 36 configurations (P =6, R ∈ {6, 8, 10}, three seeds each) for 30,500 steps (∼1B tokens at sequence length 1024). Stage 1 identifies P =6 as universally best and provides per-scale R rankings; Stage 2 tests both at full training. Stage 1's per-scale best-R rankings reverse at Stage 2: Stage 1 ranks R=8 best at d ∈ {512, 768, 1024} and R=6 best at d=256, while Stage 2 finds R=6 best at d ≥ 512 and R=8 best at d=256 -flipping all four per-scale predictions. Variable-R inference at the best d=1024 model further shows that running more iterations than trained consistently degrades performance on HellaSwag, LAMBADA, and PIQA, a clean negative result for test-time compute scaling under our recipe. We attribute these reversals to the LTI gate settling into an aggressive-discard regime (ϱ R ≈ 0.16 at d=1024, R=10) and to a text-only corpus that may lack iterationrewarding content. At the binding d=1024 parameter-parity comparison, CART loses to a parameter-matched Dense baseline by 1-2% on stored-parameter parity and by ∼ 10% on effective-parameter parity. Diagnostic ablations decompose the larger effective-parameter gap into ∼ 5% from weight sharing and a residual ∼ 5% from the heterogeneous prelude → anchor → core → coda framing itself; the recurrent-core machinery components are individually vestigial. * Code, training scripts, and the experiment database are available at https://github.com/ccapps42/ CART.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2606.01495")
get_code_for_paper("2606.01495")
have("2606.01495")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2606.01495.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2606.01495.svg)](https://syntology.ai/paper/2606.01495)

Connect an agent — have() is free.