SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2402.14433 · 2024

A Language Model's Guide Through Latent Space

arXiv · PDF · Open in the Atlas

Code that ran

11 of 11 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 2 honoured the contract we drafted, 1 violated it and 0 have no contract verdict; 8 failures of our instrument; 0 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2402.14433.json · badge: /api/ran/2402.14433.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
dvruette/concept-guidance canonical (classifier label)
classifier basis: census
11 ran · 0 no recorded run
FunctionStatusWhere it lives
apply_linear_guidance Ran
ran; our contract draft was wrong, not the code
dvruette/concept-guidance/concept_guidance/patching.py
code served (permissive licence) · get_code("2e97df5250761eae")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 3, dtype family float, shape (2, 3, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Create small tensors
# x: shape (batch=2, sequence=3, features=4) - the input tensor
x = torch.randn(2, 3, 4, dtype=torch.float32)

# w: shape (batch=2, sequence=3, features=4) - the guidance direction tensor
# Must have same shape as x for broadcasting in the function
w = torch.randn(2, 3, 4, dtype=torch.float32)

ARGS = {
    "x": x,
    "w": w,
    "guidance_scale": 1.5,
    "only_last": False,
    "normalize": True,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 3, dtype family float, shape (2, 3, 4) — shape (2, 3, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
flatten_homogeneous Ran
ran; our contract draft was wrong, not the code
dvruette/concept-guidance/concept_guidance/models/utils.py
code served (permissive licence) · get_code("25673cb24fdc6737")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# X: torch.Tensor - reshaped from (batch, seq_len, *features) to (-1, *features)
# Batch size: 2, sequence length: 3, feature dims: 4, 5
X = torch.randn(2, 3, 4, 5)

# y: torch.Tensor | None - labels for each batch element, will be expanded and reshaped
# Should have shape (batch,) = (2,)
y = torch.tensor([0, 1])

ARGS = {
    "X": X,
    "y": y,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
flatten_inhomogeneous Ran
ran; our contract draft was wrong, not the code
dvruette/concept-guidance/concept_guidance/models/utils.py
code served (permissive licence) · get_code("e24f4c1f7af2015d")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Construct a list of inhomogeneous tensors (different batch sizes, same feature dim)
X = [
    torch.randn(2, 4),  # batch size 2, features 4
    torch.randn(3, 4),  # batch size 3, features 4
    torch.randn(1, 4),  # batch size 1, features 4
]

# Construct labels corresponding to each tensor in X
y = torch.tensor([0, 1, 2], dtype=torch.long)

ARGS = {
    "X": X,
    "y": y,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
get_tokens Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
dvruette/concept-guidance/concept_guidance/activations.py
code served (permissive licence) · get_code("b168d5fa90d9b20d")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned tuple, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import copy

torch.manual_seed(0)

class MockTokenizer:
    def __init__(self):
        self.vocab_size = 100
    
    def __call__(self, text, add_special_tokens=False, return_tensors=None, truncation=False):
        # Simulate tokenization by returning input_ids
        # Length varies based on text content for realism
        if isinstance(text, list):
            text = text[0] if text else ""
        token_count = max(5, len(text.split()) + 2)
        input_ids = torch.arange(token_count, dtype=torch.long).unsqueeze(0)
        result = {'input_ids': input_ids}
        if return_tensors == 'pt':
            result['attention_mask'] = torch.ones_like(input_ids)
        return result
    
    def apply_chat_template(self, messages, tokenize=False):
        # Simulate chat template formatting
        result = ""
        for msg in messages:
            result += f"[{msg['role']}] {msg['content']} "
        return result.strip()

tokenizer = MockTokenizer()

message = {
    'prompt': 'What is the capital of France?',
    'completion': 'Paris is the capital of France.',
    'conversation_history': [
        {'role': 'user', 'content': 'Hello'},
        {'role': 'assistant', 'content': 'Hi there'}
    ]
}

system_prompt = 'You are a helpful assistant.'
max_length = 1024
max_assistant_tokens = 32
truncation = True

ARGS = {
    'tokenizer': tokenizer,
    'message': message,
    'system_prompt': system_prompt,
    'max_length': max_length,
    'max_assistant_tokens': max_assistant_tokens,
    'truncation': truncation
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch, transformers — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple, ndim 1, dtype family float — a tuple output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
guidance_patch Ran
ran; our contract draft was wrong, not the code
dvruette/concept-guidance/concept_guidance/patching.py
code served (permissive licence) · get_code("6bb2253db73fb4db")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 3, dtype family float, shape (2, 5, 4) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Create a small mock module with an old_forward method
class MockModule(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(4, 4)
    
    def old_forward(self, x):
        return self.linear(x)

# Create instances and tensors
mock_self = MockModule()

# vectors: shape [num_layers, vector_dim]
# layer_idx will index into this, so we need at least 3 layers
vectors = torch.randn(3, 4)

# layer_idx: integer to index into vectors
layer_idx = 1

# x: input tensor that gets passed to old_forward
# Shape: [batch_size, seq_len, hidden_dim]
x = torch.randn(2, 5, 4)

# guidance_scale: float
guidance_scale = 2.0

# normalize: bool
normalize = True

# only_last: bool
only_last = False

ARGS = {
    "self": mock_self,
    "vectors": vectors,
    "layer_idx": layer_idx,
    "x": x,
    "guidance_scale": guidance_scale,
    "normalize": normalize,
    "only_last": only_last,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 3, dtype family float, shape (2, 5, 4) — shape (2, 5, 4) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
is_homogeneous Ran
ran, violated the contract we drafted
dvruette/concept-guidance/concept_guidance/models/utils.py
code served (permissive licence) · get_code("3c791b0565e25485")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned bool, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Test case 1: list of tensors with homogeneous shapes
tensor1 = torch.randn(2, 4)
tensor2 = torch.randn(2, 4)
tensor3 = torch.randn(2, 4)
list_homogeneous = [tensor1, tensor2, tensor3]

# Test case 2: list of tensors with heterogeneous shapes
tensor4 = torch.randn(2, 4)
tensor5 = torch.randn(3, 5)
list_heterogeneous = [tensor4, tensor5]

# Test case 3: single tensor (not a list)
single_tensor = torch.randn(2, 4)

# Use the homogeneous list as the primary test case
ARGS = {
    "X": list_homogeneous
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type bool, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
layer_wise_accuracy Ran
ran, honoured the contract we drafted
dvruette/concept-guidance/concept_guidance/metrics.py
code served (permissive licence) · get_code("0d3c1119d8865567")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family float, shape (4,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# Construct ys_pred and ys_true
# The function iterates over these with zip and expects:
# - y_pred to be a tensor that can be compared with > 0.5
# - y_true to be a tensor that can be compared with .eq()
# - Both should support .shape[0] to check batch size > 0
# - The result of (y_pred > 0.5).eq(y_true).float().mean(dim=0) suggests
#   y_pred and y_true are 2D tensors (batch_size, num_features)

# Create a list of batch pairs
ys_pred = [
    torch.rand(2, 4),  # batch_size=2, num_features=4
    torch.rand(3, 4),  # batch_size=3, num_features=4
]

ys_true = [
    torch.randint(0, 2, (2, 4)).float(),  # batch_size=2, num_features=4
    torch.randint(0, 2, (3, 4)).float(),  # batch_size=3, num_features=4
]

ARGS = {
    "ys_pred": ys_pred,
    "ys_true": ys_true,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ndarray, ndim 1, dtype family float, shape (4,) — shape (4,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
load_weights Ran
ran, honoured the contract we drafted
dvruette/concept-guidance/concept_guidance/patching.py
code served (permissive licence) · get_code("e96d4bb13179f09a")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned Tensor, ndim 2, dtype family float, shape (4, 3) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import tempfile
from pathlib import Path
from safetensors.torch import save_file

torch.manual_seed(0)

# Create a temporary directory and a valid safetensors file with a 'W' key
temp_dir = tempfile.mkdtemp()
temp_path = Path(temp_dir) / "weights.safetensors"

# Create a small tensor and save it with the required 'W' key
weights_dict = {"W": torch.randn(4, 3)}
save_file(weights_dict, temp_path)

ARGS = {
    "path": temp_path,
    "device": "cpu"
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
safetensors, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float, shape (4, 3) — shape (4, 3) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
parse_dtype Ran
ran; our contract draft was wrong, not the code
dvruette/concept-guidance/concept_guidance/utils.py
code served (permissive licence) · get_code("517b074ae9e90122")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned dtype. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# The function parse_dtype takes a single parameter 'dtype' which is a string
# It checks the dtype against 'float16', 'float32', or 'float64' and returns the corresponding torch dtype
# If dtype doesn't match any of these, it raises a ValueError

ARGS = {
    'dtype': 'float32'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type dtype — a dtype output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a dtype output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
render_example Ran
ran; our contract draft was wrong, not the code
dvruette/concept-guidance/concept_guidance/eval/open_assistant.py
code served (permissive licence) · get_code("d2ce6748b8f7dcf2")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

ARGS = {
    "prompt": "This is a sample prompt for testing the render_example function with some text content that will be processed.",
    "completion": "This is the completion text that follows the prompt and provides a response or continuation.",
    "label": 1
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
truncate Ran
ran; our contract draft was wrong, not the code
dvruette/concept-guidance/concept_guidance/eval/open_assistant.py
code served (permissive licence) · get_code("2b8ac1277f79833e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned str. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch

torch.manual_seed(0)

# The function truncate takes:
# - text: str - a string to potentially truncate
# - max_chars: int - maximum number of characters allowed (default 200)

# Test case 1: text shorter than max_chars
text_short = "Hello world"
max_chars_short = 20

# Test case 2: text longer than max_chars
text_long = "The quick brown fox jumps over the lazy dog. This is a longer text that will need to be truncated because it exceeds the maximum character limit we have set for this example."
max_chars_long = 50

ARGS = {
    "text": text_long,
    "max_chars": max_chars_long
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type str — a str output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a str output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block

Also present in other papers' code

Of 11 samples lifted from this paper's linked repositories, 11 carry a function identity and 0 do not. Of the 11 distinct functions with an identity, 0 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 0 later) and 11 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Concept guidance has emerged as a cheap and simple way to control the behavior of language models by probing their hidden representations for concept vectors and using them to perturb activations at inference time. While the focus of previous work has largely been on truthfulness, in this paper we extend this framework to a richer set of concepts such as appropriateness, humor, creativity and quality, and explore to what degree current detection and guidance strategies work in these challenging settings. To facilitate evaluation, we develop a novel metric for concept guidance that takes into account both the success of concept elicitation as well as the potential degradation in fluency of the guided model. Our extensive experiments reveal that while some concepts such as truthfulness more easily allow for guidance with current techniques, novel concepts such as appropriateness or humor either remain difficult to elicit, need extensive tuning to work, or even experience confusion. Moreover, we find that probes with optimal detection accuracies do not necessarily make for the optimal guides, contradicting previous observations for truthfulness. Our work warrants a deeper investigation into the interplay between detectability, guidability, and the nature of the concept, and we hope that our rich experimental test-bed for guidance research inspires stronger follow-up approaches.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2402.14433")
get_code_for_paper("2402.14433")
have("2402.14433")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2402.14433.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2402.14433.svg)](https://syntology.ai/paper/2402.14433)

Connect an agent — have() is free.