SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2410.04468 · ICLR · 2025

Revisiting In-context Learning Inference Circuit in Large Language Models

Hakaze Cho, Mariko Kato, Yoshihiro Sakai, Naoya Inoue

arXiv · PDF · Open in the Atlas

Code that ran

11 of 15 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 7 honoured the contract we drafted, 0 violated it and 0 have no contract verdict; 4 failures of our instrument; 4 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2410.04468.json · badge: /api/ran/2410.04468.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
hc495/ICL_Circuit canonical (classifier label)
classifier basis: census
11 ran · 4 no recorded run
FunctionStatusWhere it lives
ICL_inference_to_hidden_states Ran
ran, honoured the contract we drafted
hc495/ICL_Circuit/util/inference.py
code served (permissive licence) · get_code("6d2ba9201e0266c1")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 3, dtype family float; no output shape is stated. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
import numpy as np

torch.manual_seed(0)

class MockTokenizer:
    def __call__(self, text, return_tensors=None):
        # Return a simple mock result with input_ids
        result = MockTokenizerOutput()
        # Create a small tensor: batch_size=1, seq_len=3
        result.input_ids = torch.randint(0, 100, (1, 3))
        return result

class MockTokenizerOutput:
    def __init__(self):
        self.input_ids = None
    
    def to(self, device):
        # Return a tensor that can be moved to device
        return self.input_ids.to(device)

class MockModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.device_attr = 'cpu'
        self.linear = nn.Linear(4, 8)
    
    @property
    def device(self):
        return self.device_attr
    
    def forward(self, input_ids, output_hidden_states=False):
        # Simulate model forward pass
        # input_ids shape: (batch, seq_len)
        batch_size, seq_len = input_ids.shape
        
        # Create mock hidden states for multiple layers
        # Each layer outputs (batch_size, seq_len, hidden_dim)
        num_layers = 3
        hidden_dim = 8
        
        result = MockModelOutput()
        hidden_states = []
        
        for layer_idx in range(num_layers):
            # Create hidden state: (1, 3, 8) for each layer
            layer_hidden = torch.randn(batch_size, seq_len, hidden_dim)
            hidden_states.append(layer_hidden)
        
        result.hidden_states = hidden_states
        return result

class MockModelOutput:
    def __init__(self):
        self.hidden_states = None

# Create instances
mock_model = MockModel()
mock_tokenizer = MockTokenizer()

# Create prompts list
prompts = [
    "This is the first prompt for in-context learning.",
    "This is the second prompt for in-context learning."
]

ARGS = {
    "model": mock_model,
    "tokenizer": mock_tokenizer,
    "prompts": prompts
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch, tqdm — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 3, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ICL_inference_to_hidden_states_transposed Ran
ran; our contract draft was wrong, not the code
hc495/ICL_Circuit/util/inference.py
code served (permissive licence) · get_code("0e4a9d80be097eab")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
import numpy as np

torch.manual_seed(0)

# Create a minimal tokenizer that satisfies the interface
class MinimalTokenizer:
    def __call__(self, text, return_tensors=None):
        # Return a simple object with input_ids attribute
        class TokenizerOutput:
            def __init__(self):
                self.input_ids = torch.randint(0, 100, (1, 5))
            
            def to(self, device):
                self.input_ids = self.input_ids.to(device)
                return self.input_ids
        
        return TokenizerOutput()

# Create a minimal model that satisfies the interface
class MinimalModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.device_attr = 'cpu'
        # Create 3 layers, each producing hidden states of dimension 4
        self.layer1 = nn.Linear(5, 4)
        self.layer2 = nn.Linear(4, 4)
        self.layer3 = nn.Linear(4, 4)
    
    @property
    def device(self):
        return self.device_attr
    
    def forward(self, input_ids, output_hidden_states=False):
        # Simulate model forward pass with hidden states
        batch_size = input_ids.shape[0]
        seq_len = input_ids.shape[1]
        
        # Convert input_ids to embeddings (dummy)
        x = torch.randn(batch_size, seq_len, 5)
        
        hidden_states = [x]  # Initial embedding as first hidden state
        
        # Generate hidden states for 3 layers
        x_current = x.mean(dim=1)  # Pool to get (batch, 5)
        h1 = self.layer1(x_current).unsqueeze(1)  # (batch, 1, 4)
        hidden_states.append(h1)
        
        h2 = self.layer2(h1.squeeze(1)).unsqueeze(1)  # (batch, 1, 4)
        hidden_states.append(h2)
        
        h3 = self.layer3(h2.squeeze(1)).unsqueeze(1)  # (batch, 1, 4)
        hidden_states.append(h3)
        
        class ModelOutput:
            pass
        
        result = ModelOutput()
        result.hidden_states = hidden_states
        return result

model = MinimalModel()
tokenizer = MinimalTokenizer()
prompts = ["Hello world", "How are you"]

ARGS = {
    "model": model,
    "tokenizer": tokenizer,
    "prompts": prompts
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch, tqdm — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list — a list output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
encoder_inference_to_feature Ran
ran, honoured the contract we drafted
hc495/ICL_Circuit/util/inference.py
code served (permissive licence) · get_code("03f5ae8eefe2ed8f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 3, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

class StandInTokenizer:
    def __call__(self, query, return_tensors=None):
        # Return a simple object with input_ids attribute
        class TokenizerOutput:
            def __init__(self):
                self.input_ids = torch.randint(0, 100, (1, 5))
        return TokenizerOutput()

class StandInModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.device_attr = torch.device('cpu')
        self.linear = nn.Linear(5, 8)
    
    @property
    def device(self):
        return self.device_attr
    
    def forward(self, input_ids, output_hidden_states=False):
        # Return a simple result object with pooler_output attribute
        batch_size = input_ids.shape[0]
        hidden_state = self.linear(input_ids.float())
        
        class ModelOutput:
            def __init__(self, pooler_output):
                self.pooler_output = pooler_output
        
        # pooler_output should be indexable with [-1]
        pooler_out = torch.randn(batch_size, 8)
        return ModelOutput(pooler_output=[pooler_out])

model = StandInModel()
tokenizer = StandInTokenizer()
queries = ["query one", "query two", "query three", "query four", "query five"]

ARGS = {
    "model": model,
    "tokenizer": tokenizer,
    "queries": queries
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch, tqdm — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 3, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
find_tokenized_label_word Ran
ran, honoured the contract we drafted
hc495/ICL_Circuit/util/ablation_study.py
code served (permissive licence) · get_code("dc2a498623d25b35")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 2, dtype family int; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

class MockTokenizer:
    def __call__(self, text):
        # Simple mock tokenizer that returns input_ids based on text length
        # For divider " label:", return specific token ids that can be found
        if "label:" in text:
            return {'input_ids': [101, 2054, 2003, 100, 100, 102]}
        else:
            # For the prompt, return a longer sequence that contains the divider pattern
            return {'input_ids': [101, 2054, 2003, 100, 100, 102, 2054, 2003]}

class MockPromptFormer:
    def __init__(self):
        self._label_prefix = "label:"

class MockExperimentor:
    def __init__(self):
        self.prompt_former = MockPromptFormer()

tokenizer = MockTokenizer()
experimentor = MockExperimentor()
prompt = "What is [MASK] label: [MASK]"

ARGS = {
    "tokenizer": tokenizer,
    "experimentor": experimentor,
    "prompt": prompt,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 2, dtype family int — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_induction_correctness_for_single_layer Ran
ran, honoured the contract we drafted
hc495/ICL_Circuit/util/induction_evaluation.py
code served (permissive licence) · get_code("ab0facc671844bd0")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

class MockExperimentor:
    def __init__(self):
        self.demonstration_sampler = {
            0: [0, 1],
            1: [2, 3]
        }
        self._demonstration_set = [
            ("demo_0", "label_a"),
            ("demo_1", "label_b"),
            ("demo_2", "label_a"),
            ("demo_3", "label_c")
        ]
        self._test_set = [
            ("test_0", "label_a"),
            ("test_1", "label_b")
        ]
        self._label_space = ["label_a", "label_b", "label_c"]
    
    def demonstration_sampler_get(self, index):
        return self.demonstration_sampler[index]
    
    def demonstration_set(self):
        return self._demonstration_set
    
    def test_set(self):
        return self._test_set
    
    def get_label_space(self):
        return self._label_space

experimentor = MockExperimentor()

ICL_attention = {
    0: {
        0: [
            torch.tensor([0.1, 0.2, 0.3, 0.4]),
            torch.tensor([0.15, 0.25, 0.35, 0.45])
        ]
    },
    1: {
        0: [
            torch.tensor([0.2, 0.3, 0.4, 0.5]),
            torch.tensor([0.25, 0.35, 0.45, 0.55])
        ]
    }
}

ARGS = {
    "ICL_attention": ICL_attention,
    "experimentor": experimentor,
    "sample_index": 0,
    "layer": 0
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_induction_magnitude_for_single_layer Ran
ran; our contract draft was wrong, not the code
hc495/ICL_Circuit/util/induction_evaluation.py
code served (permissive licence) · get_code("36830c7e7fd81df1")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

class DemonstrationSampler:
    def __init__(self):
        self.samples = {
            0: [0, 1],  # sample_index 0 has demonstrations at indices 0, 1
            1: [2, 3]
        }
    
    def __getitem__(self, key):
        return self.samples[key]

class Experimentor:
    def __init__(self):
        self.demonstration_sampler = DemonstrationSampler()
        self.label_space = ['cat', 'dog', 'bird']
        self.demo_set = [
            ('demo_input_0', 'cat'),
            ('demo_input_1', 'dog'),
            ('demo_input_2', 'bird'),
            ('demo_input_3', 'cat')
        ]
        self.test_set_data = [
            ('test_input_0', 'dog'),
            ('test_input_1', 'cat')
        ]
    
    def get_label_space(self):
        return self.label_space
    
    def demonstration_set(self):
        return self.demo_set
    
    def test_set(self):
        return self.test_set_data

experimentor = Experimentor()

# ICL_attention: indexed as ICL_attention[sample_index][layer][head_index][position]
# For sample_index=0, layer=0, we need heads with attention at positions 2*i+1 for each demo
# 2 demonstrations means we need attention at positions 1 and 3
ICL_attention = [
    # sample_index 0
    [
        # layer 0
        [
            # head 0: attention scores across positions
            torch.tensor([0.1, 0.5, 0.2, 0.6, 0.1]),  # positions for 2 demos (1, 3)
            # head 1
            torch.tensor([0.2, 0.4, 0.3, 0.7, 0.2])
        ]
    ],
    # sample_index 1
    [
        # layer 0
        [
            # head 0
            torch.tensor([0.15, 0.55, 0.25, 0.65, 0.15]),
            # head 1
            torch.tensor([0.25, 0.45, 0.35, 0.75, 0.25])
        ]
    ]
]

ARGS = {
    'ICL_attention': ICL_attention,
    'experimentor': experimentor,
    'sample_index': 0,
    'layer': 0
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
kernel_alignment Ran
ran; our contract draft was wrong, not the code
hc495/ICL_Circuit/util/kernel_alignment.py
code served (permissive licence) · get_code("61a2e6dfc5a9fe1f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# Create two similarity graph matrices
# The function expects:
# - simGraph_1 and simGraph_2: array-like objects where each row can be sorted
# - k: integer for top-k selection
# We'll create 2D numpy arrays where each row represents similarity scores

np.random.seed(0)

# Create simGraph_1: shape (5, 10) - 5 samples, 10 similarity scores each
simGraph_1 = np.random.rand(5, 10)

# Create simGraph_2: shape (5, 10) - same structure for comparison
simGraph_2 = np.random.rand(5, 10)

# k is already defined with default value 64, but we'll use a smaller value
# that fits our small graph size
k = 3

ARGS = {
    "simGraph_1": simGraph_1,
    "simGraph_2": simGraph_2,
    "k": k
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
load_data_from_StaICC_experimentor Ran
ran; our contract draft was wrong, not the code
hc495/ICL_Circuit/util/load_model_and_data.py
code served (permissive licence) · get_code("638cfdf35bdeb6ba")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

class MockPromptFormer:
    def __init__(self):
        self._label_space = ['positive', 'negative', 'neutral']
        self._label_prefix = 'Label: '

class MockLabelSpace:
    def __init__(self):
        self.labels = ['positive', 'negative', 'neutral']
    
    def index(self, label):
        return self.labels.index(label)

class MockQuerySet:
    def __init__(self):
        self._label_space = MockLabelSpace()
        self.data = [
            (([1, 2, 3], ), 'positive'),
            (([4, 5, 6], ), 'negative'),
        ]
    
    def __len__(self):
        return len(self.data)
    
    def __getitem__(self, idx):
        return self.data[idx]

class MockExperimentor:
    def __init__(self):
        self.prompt_former = MockPromptFormer()
        self._test_set = MockQuerySet()
        self._prompts = [
            'What is the sentiment? ',
            'Classify this text: ',
        ]
    
    def test_set(self):
        return self._test_set
    
    def prompt_set(self):
        return self._prompts

experimentor = MockExperimentor()

ARGS = {
    'experimentor': experimentor,
    'prompt_cut': 'none',
    'target_label_correction': True,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
overlap Ran
ran, honoured the contract we drafted
hc495/ICL_Circuit/util/kernel_alignment.py
code served (permissive licence) · get_code("5d93a88b24afc491")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned int, ndim 0, dtype family int, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# The function overlap(a, b) iterates through items in 'a' and checks if each
# item is in 'b', counting matches. Both parameters should be iterables.
# We can use lists, tuples, sets, or any iterable containing comparable items.

# Create simple iterables with some overlapping and non-overlapping elements
a = [1, 2, 3, 4, 5]
b = [3, 4, 5, 6, 7]

ARGS = {
    "a": a,
    "b": b
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type int, ndim 0, dtype family int, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
sim_graph Ran
ran, honoured the contract we drafted
hc495/ICL_Circuit/util/kernel_alignment.py
code served (permissive licence) · get_code("81967b2af6f55803")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# Create a small feature matrix for testing
# Shape: (5, 4) - 5 samples, 4-dimensional features
np.random.seed(0)
features = np.random.randn(5, 4).astype(np.float32)

ARGS = {
    "features": features
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, tqdm — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 2, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
tokenized_length Ran
ran, honoured the contract we drafted
hc495/ICL_Circuit/util/induction_evaluation.py
code served (permissive licence) · get_code("3a45ae8b2ab1d6e9")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned int, ndim 0, dtype family int, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from transformers import AutoTokenizer

torch.manual_seed(0)

# Create a minimal stand-in tokenizer that has the __call__ method
class MinimalTokenizer:
    def __call__(self, prompt):
        # Return a dict with 'input_ids' key containing a list of token IDs
        # Length depends on the prompt length (simple heuristic: ~1 token per word)
        tokens = [101] + [1000 + i for i in range(len(prompt.split()))] + [102]
        return {'input_ids': tokens}

tokenizer = MinimalTokenizer()
prompt = "Hello world this is a test"

ARGS = {
    'tokenizer': tokenizer,
    'prompt': prompt
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type int, ndim 0, dtype family int, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Masked_ICL_inference Not yet run
unverified (harvested, no recorded run)
hc495/ICL_Circuit/util/ablation_study.py
code served (permissive licence) · get_code("8ac576595ccbc620")
How it ran

no invocation recorded.

head_ablation Not yet run
unverified (harvested, no recorded run)
hc495/ICL_Circuit/util/ablation_study.py
code served (permissive licence) · get_code("ab89dd08b607b43c")
How it ran

no invocation recorded.

load_ICL_model Not yet run
unverified (harvested, no recorded run)
hc495/ICL_Circuit/util/load_model_and_data.py
code served (permissive licence) · get_code("f906d6d0f7a495ad")
How it ran

no invocation recorded.

load_encode_model Not yet run
unverified (harvested, no recorded run)
hc495/ICL_Circuit/util/load_model_and_data.py
code served (permissive licence) · get_code("fdf0bd39d2cf5fed")
How it ran

no invocation recorded.

Also present in other papers' code

Of 15 samples lifted from this paper's linked repositories, 15 carry a function identity and 0 do not. Of the 15 distinct functions with an identity, 0 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 0 later) and 15 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

In-context Learning (ICL) is an emerging few-shot learning paradigm on Language Models (LMs) with inner mechanisms un-explored. There are already existing works describing the inner processing of ICL, while they struggle to capture all the inference phenomena in large language models. Therefore, this paper proposes a comprehensive circuit to model the inference dynamics and try to explain the observed phenomena of ICL. In detail, we divide ICL inference into 3 major operations: (1) Input Text Encode: LMs encode every input text (in the demonstrations and queries) into linear representation in the hidden states with sufficient information to solve ICL tasks. (2) Semantics Merge: LMs merge the encoded representations of demonstrations with their corresponding label tokens to produce joint representations of labels and demonstrations. (3) Feature Retrieval and Copy: LMs search the joint representations of demonstrations similar to the query representation on a task subspace, and copy the searched representations into the query. Then, language model heads capture these copied label representations to a certain extent and decode them into predicted labels. Through careful measurements, the proposed inference circuit successfully captures and unifies many fragmented phenomena observed during the ICL process, making it a comprehensive and practical explanation of the ICL inference process. Moreover, ablation analysis by disabling the proposed steps seriously damages the ICL performance, suggesting the proposed inference circuit is a dominating mechanism. Additionally, we confirm and list some bypass mechanisms that solve ICL tasks in parallel with the proposed circuit.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2410.04468")
get_code_for_paper("2410.04468")
have("2410.04468")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2410.04468.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2410.04468.svg)](https://syntology.ai/paper/2410.04468)

Connect an agent — have() is free.