SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2402.07307 · NeurIPS · 2024

Self-Calibrating Conformal Prediction

Ahmed Alaa, Lars Van Der Laan

arXiv · PDF · Open in the Atlas

Code that ran

23 of 25 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (4 of them constructed an object rather than computing a result; 2 honoured the contract we drafted, 0 violated it and 4 have no contract verdict; 17 failures of our instrument; 2 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2402.07307.json · badge: /api/ran/2402.07307.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
larsvanderlaan/selfcalibratingconformal canonical (classifier label)
classifier basis: classifier_2025
20 ran · 2 no recorded run
Larsvanderlaan/SelfCalibratingConformal canonical (classifier label)
classifier basis: census
3 ran · 0 no recorded run
FunctionStatusWhere it lives
HistogramConfig Ran
ran on a synthesized input · constructed an object; no result was computed
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("f2d9b65a76a0a98d")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, HistogramConfig: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from dataclasses import dataclass

torch.manual_seed(0)

@dataclass(frozen=True)
class HistogramConfig:
    num_bin: int = 5
    binning_method: str = 'quantile'

    def validate(self) -> None:
        if self.num_bin < 2:
            raise ValueError('num_bin must be at least 2.')
        if self.binning_method not in {'quantile', 'fixed'}:
            raise ValueError("binning_method must be 'quantile' or 'fixed'.")

ARGS = {
    'num_bin': 5,
    'binning_method': 'quantile',
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type HistogramConfig — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a HistogramConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
PredictionAdapter Ran
ran on a synthesized input · constructed an object; no result was computed
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("d0a618b0a8c89ef2")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, PredictionAdapter: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np
from dataclasses import dataclass
from typing import Any, Protocol, runtime_checkable

@runtime_checkable
class PredictorProtocol(Protocol):
    def predict(self, x: Any) -> Any:
        ...

def ensure_1d_array(values) -> np.ndarray:
    array = np.asarray(values, dtype=float)
    if array.ndim == 0:
        array = array.reshape(1)
    return np.ravel(array)

@dataclass(frozen=True)
class PredictionAdapter:
    predictor: Any

    def __call__(self, x: Any) -> np.ndarray:
        if callable(self.predictor) and (not hasattr(self.predictor, 'predict')):
            values = self.predictor(x)
        elif isinstance(self.predictor, PredictorProtocol):
            values = self.predictor.predict(x)
        elif hasattr(self.predictor, 'predict'):
            values = self.predictor.predict(x)
        else:
            raise TypeError('Predictor must be a callable or expose a .predict method.')
        return ensure_1d_array(values)


# Test with a simple callable predictor
def simple_predictor(x):
    return np.array([1.0, 2.0, 3.0])

# Test with a class that has a predict method
class ModelPredictor:
    def predict(self, x: Any) -> Any:
        return np.array([4.0, 5.0])

# Test with a callable class (implements __call__)
class CallablePredictor:
    def __call__(self, x: Any) -> Any:
        return np.array([6.0, 7.0, 8.0, 9.0])

ARGS = {
    "predictor": simple_predictor
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type PredictionAdapter — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a PredictionAdapter output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
RegressionAlgoConfig Ran
ran on a synthesized input · constructed an object; no result was computed
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("9b87b253916c3a4f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, RegressionAlgoConfig: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
from dataclasses import dataclass

torch.manual_seed(0)

ARGS = {
    'num_bin_predictor': 100,
    'num_bin_y': 80,
    'binning_method': 'quantile',
    'interpolation_grid_size': 1000,
    'smooth_kind': 'linear',
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type RegressionAlgoConfig — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a RegressionAlgoConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
RegressionFitState Ran
ran on a synthesized input · constructed an object; no result was computed
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("b8c92398dd17d5da")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, RegressionFitState: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np
from dataclasses import dataclass
import pandas as pd

@dataclass
class RegressionFitState:
    prediction_grid: np.ndarray
    calibrated_prediction: np.ndarray
    venn_abers_bounds: np.ndarray
    interval_bounds: np.ndarray
    raw_venn_paths: np.ndarray

    def to_frame(self) -> pd.DataFrame:
        return pd.DataFrame({'prediction_uncal': list(self.prediction_grid), 'prediction_cal': list(self.calibrated_prediction), 'prediction_venn_abers': list(self.venn_abers_bounds), 'prediction_interval': list(self.interval_bounds)})


ARGS = {
    'prediction_grid': np.array([1.2, 2.3, 3.4, 4.5]),
    'calibrated_prediction': np.array([1.1, 2.2, 3.3, 4.4]),
    'venn_abers_bounds': np.array([[1.0, 1.3], [2.0, 2.5], [3.0, 3.6], [4.0, 4.7]]),
    'interval_bounds': np.array([[0.9, 1.4], [1.9, 2.6], [2.9, 3.7], [3.9, 4.8]]),
    'raw_venn_paths': np.array([[0.1, 0.2, 0.3], [0.15, 0.25, 0.35], [0.12, 0.22, 0.32], [0.18, 0.28, 0.38]])
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, pandas — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type RegressionFitState — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a RegressionFitState output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
_coerce_dataclass Ran
ran; our contract draft was wrong, not the code
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("70511a2064961b91")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned SampleConfig. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import warnings
from dataclasses import dataclass, fields
from typing import Any, Mapping, Type, TypeVar

T = TypeVar('T')

def _coerce_dataclass(value: Any, cls: Type[T], *, warning_name: str, default: T) -> T:
    if value is None:
        return default
    if isinstance(value, cls):
        return value
    if isinstance(value, Mapping):
        warnings.warn(f'{warning_name} dictionaries remain supported for backward compatibility, but passing {cls.__name__} is preferred.', DeprecationWarning, stacklevel=3)
        allowed = {field.name for field in fields(cls)}
        filtered = {key: val for key, val in value.items() if key in allowed}
        return cls(**filtered)
    raise TypeError(f'{warning_name} must be a {cls.__name__}, mapping, or None.')

@dataclass
class SampleConfig:
    learning_rate: float = 0.001
    batch_size: int = 32
    hidden_dim: int = 128

ARGS = {
    'value': {
        'learning_rate': 0.002,
        'batch_size': 64,
        'hidden_dim': 256
    },
    'cls': SampleConfig,
    'warning_name': 'model_config',
    'default': SampleConfig(learning_rate=0.001, batch_size=32, hidden_dim=128)
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type SampleConfig — a SampleConfig output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a SampleConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
_isotonic_block_fit Ran
ran; our contract draft was wrong, not the code
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("c4b6224294c04c80")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned function. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# Define a reducer function that will be passed as the 'reducer' parameter
# The function uses: reducer(np.array(...)) - it calls reducer with a numpy array
def mean_reducer(values):
    """Simple reducer that computes the mean of values"""
    return np.mean(values)

# Create sample data for x and y
x_data = np.array([1.0, 2.0, 3.0, 4.0])
y_data = np.array([2.0, 1.0, 3.0, 2.5])

ARGS = {
    'x': x_data,
    'y': y_data,
    'reducer': mean_reducer
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type function — a function output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a function output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
_step_transform Ran
ran; our contract draft was wrong, not the code
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("ffc7bda663953207")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned function. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# Create a small grid of evenly spaced values
grid = np.array([0.0, 1.0, 2.0, 3.0, 4.0])

# Create corresponding values for each grid point
values = np.array([10.0, 20.0, 30.0, 40.0, 50.0])

ARGS = {
    "grid": grid,
    "values": values,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type function — a function output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a function output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
_weighted_quantile Ran
ran; our contract draft was wrong, not the code
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("541c619537506f29")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# Construct test arguments for _weighted_quantile function
values = np.array([1.5, 2.3, 3.7, 4.2, 5.1, 6.8])
q = 0.5

ARGS = {
    "values": values,
    "q": q,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
adapt_predictor Ran
ran; our contract draft was wrong, not the code
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("628015812b19addd")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned PredictionAdapter. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np
from dataclasses import dataclass
from typing import Any, Callable, Protocol, runtime_checkable

@runtime_checkable
class PredictorProtocol(Protocol):
    def predict(self, x: Any) -> Any:
        ...

def ensure_1d_array(values) -> np.ndarray:
    array = np.asarray(values, dtype=float)
    if array.ndim == 0:
        array = array.reshape(1)
    return np.ravel(array)

@dataclass(frozen=True)
class PredictionAdapter:
    predictor: Any

    def __call__(self, x: Any) -> np.ndarray:
        if callable(self.predictor) and (not hasattr(self.predictor, 'predict')):
            values = self.predictor(x)
        elif isinstance(self.predictor, PredictorProtocol):
            values = self.predictor.predict(x)
        elif hasattr(self.predictor, 'predict'):
            values = self.predictor.predict(x)
        else:
            raise TypeError('Predictor must be a callable or expose a .predict method.')
        return ensure_1d_array(values)

def adapt_predictor(predictor: Any) -> PredictionAdapter:
    return PredictionAdapter(predictor=predictor)


class SimpleCallablePredictor:
    """A simple callable predictor without a predict method."""
    def __call__(self, x):
        return np.array([1.0, 2.0, 3.0])


class SimplePredictorWithMethod:
    """A predictor with a predict method."""
    def predict(self, x):
        return np.array([4.0, 5.0])


def simple_function_predictor(x):
    """A simple function that acts as a predictor."""
    return np.array([6.0, 7.0, 8.0, 9.0])


ARGS = {
    "predictor": SimpleCallablePredictor()
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type PredictionAdapter — a PredictionAdapter output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a PredictionAdapter output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
calibrator_histogram Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("5ed9f5202825f1ce")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned function. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# Set up random seed for reproducibility
np.random.seed(0)

# Create synthetic prediction scores and binary labels
# f: model predictions (continuous values between 0 and 1)
# y: ground truth labels (binary: 0 or 1)
f = np.array([0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.85, 0.9], dtype=float)
y = np.array([0, 0, 0, 0, 1, 1, 1, 1, 1, 1], dtype=float)

ARGS = {
    'f': f,
    'y': y,
    'num_bin': 5,
    'binning_method': 'quantile',
    'quantile_level': None
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type function — a function output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a function output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
calibrator_isotonic Ran
ran; our contract draft was wrong, not the code
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("9138963f8018cfac")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned function. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

def ensure_1d_array(values) -> np.ndarray:
    array = np.asarray(values, dtype=float)
    if array.ndim == 0:
        array = array.reshape(1)
    return np.ravel(array)

def _isotonic_block_fit(x: np.ndarray, y: np.ndarray, reducer):
    order = np.argsort(x, kind='mergesort')
    x_sorted = ensure_1d_array(x)[order]
    y_sorted = ensure_1d_array(y)[order]
    blocks: list[dict[str, np.ndarray | float]] = []
    for x_value, y_value in zip(x_sorted, y_sorted):
        blocks.append({'x_left': float(x_value), 'x_right': float(x_value), 'values': np.array([float(y_value)]), 'prediction': float(reducer(np.array([float(y_value)])))})
        while len(blocks) >= 2 and blocks[-2]['prediction'] > blocks[-1]['prediction']:
            right = blocks.pop()
            left = blocks.pop()
            merged_values = np.concatenate([left['values'], right['values']])
            blocks.append({'x_left': left['x_left'], 'x_right': right['x_right'], 'values': merged_values, 'prediction': float(reducer(merged_values))})
    intervals = [(float(block['x_left']), float(block['x_right']), float(block['prediction'])) for block in blocks]

    def transform(x_new):
        x_new = ensure_1d_array(x_new)
        result = np.empty_like(x_new, dtype=float)
        for index, value in enumerate(x_new):
            assigned = False
            for left, right, prediction in intervals:
                if left <= value <= right:
                    result[index] = prediction
                    assigned = True
                    break
            if assigned:
                continue
            if value < intervals[0][0]:
                result[index] = intervals[0][2]
            else:
                result[index] = intervals[-1][2]
        return result
    return transform

def calibrator_isotonic(f: np.ndarray, y: np.ndarray, **kwargs):
    del kwargs
    return _isotonic_block_fit(f, y, reducer=np.mean)

f_values = np.array([0.1, 0.3, 0.2, 0.5, 0.4, 0.6], dtype=float)
y_values = np.array([0.0, 1.0, 0.0, 1.0, 1.0, 1.0], dtype=float)

ARGS = {
    'f': f_values,
    'y': y_values,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type function — a function output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a function output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
coerce_histogram_config Ran
ran; our contract draft was wrong, not the code
Larsvanderlaan/SelfCalibratingConformal/selfcalibratingconformal/configs.py
code served (permissive licence) · get_code("a5218d94fbe5bb1e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned HistogramConfig. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import warnings
from dataclasses import dataclass, fields
from typing import Any, Mapping, Type, TypeVar

T = TypeVar('T')

def _coerce_dataclass(value: Any, cls: Type[T], *, warning_name: str, default: T) -> T:
    if value is None:
        return default
    if isinstance(value, cls):
        return value
    if isinstance(value, Mapping):
        warnings.warn(f'{warning_name} dictionaries remain supported for backward compatibility, but passing {cls.__name__} is preferred.', DeprecationWarning, stacklevel=3)
        allowed = {field.name for field in fields(cls)}
        filtered = {key: val for key, val in value.items() if key in allowed}
        return cls(**filtered)
    raise TypeError(f'{warning_name} must be a {cls.__name__}, mapping, or None.')

@dataclass(frozen=True)
class HistogramConfig:
    num_bin: int = 5
    binning_method: str = 'quantile'

    def validate(self) -> None:
        if self.num_bin < 2:
            raise ValueError('num_bin must be at least 2.')
        if self.binning_method not in {'quantile', 'fixed'}:
            raise ValueError("binning_method must be 'quantile' or 'fixed'.")

def coerce_histogram_config(value: Any, *, warning_name: str) -> HistogramConfig:
    config = _coerce_dataclass(value, HistogramConfig, warning_name=warning_name, default=HistogramConfig())
    config.validate()
    return config

ARGS = {
    'value': {
        'num_bin': 10,
        'binning_method': 'quantile'
    },
    'warning_name': 'test_histogram_config'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type HistogramConfig — a HistogramConfig output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a HistogramConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
coerce_quantile_algo_config Ran
ran; our contract draft was wrong, not the code
Larsvanderlaan/SelfCalibratingConformal/selfcalibratingconformal/configs.py
code served (permissive licence) · get_code("d07d076cff2b6832")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned QuantileAlgoConfig. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import warnings
from dataclasses import dataclass, fields
from typing import Any, Mapping, Type, TypeVar

T = TypeVar('T')

def _coerce_dataclass(value: Any, cls: Type[T], *, warning_name: str, default: T) -> T:
    if value is None:
        return default
    if isinstance(value, cls):
        return value
    if isinstance(value, Mapping):
        warnings.warn(f'{warning_name} dictionaries remain supported for backward compatibility, but passing {cls.__name__} is preferred.', DeprecationWarning, stacklevel=3)
        allowed = {field.name for field in fields(cls)}
        filtered = {key: val for key, val in value.items() if key in allowed}
        return cls(**filtered)
    raise TypeError(f'{warning_name} must be a {cls.__name__}, mapping, or None.')

@dataclass(frozen=True)
class QuantileAlgoConfig:
    num_bin_predictor: int = 100
    num_bin_score: int = 120
    binning_method: str = 'quantile'
    interpolation_grid_size: int = 800
    score_range_padding: float = 0.05

    def validate(self) -> None:
        if self.num_bin_predictor < 2:
            raise ValueError('num_bin_predictor must be at least 2.')
        if self.num_bin_score < 10:
            raise ValueError('num_bin_score must be at least 10.')
        if self.interpolation_grid_size < 50:
            raise ValueError('interpolation_grid_size must be at least 50.')
        if self.binning_method not in {'quantile', 'fixed'}:
            raise ValueError("binning_method must be 'quantile' or 'fixed'.")
        if self.score_range_padding < 0:
            raise ValueError('score_range_padding must be non-negative.')

def coerce_quantile_algo_config(value: Any) -> QuantileAlgoConfig:
    config = _coerce_dataclass(value, QuantileAlgoConfig, warning_name='algo_params', default=QuantileAlgoConfig())
    config.validate()
    return config

ARGS = {
    'value': {
        'num_bin_predictor': 50,
        'num_bin_score': 100,
        'binning_method': 'quantile',
        'interpolation_grid_size': 500,
        'score_range_padding': 0.1
    }
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type QuantileAlgoConfig — a QuantileAlgoConfig output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a QuantileAlgoConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
coerce_regression_algo_config Ran
ran; our contract draft was wrong, not the code
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("143bbabb14851e55")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned RegressionAlgoConfig. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
from typing import Any, Mapping

torch.manual_seed(0)

# Test case 1: None value (should use default)
ARGS = {
    'value': None
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type RegressionAlgoConfig — a RegressionAlgoConfig output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a RegressionAlgoConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
coerce_regression_algo_config Ran
ran; our contract draft was wrong, not the code
Larsvanderlaan/SelfCalibratingConformal/selfcalibratingconformal/configs.py
code served (permissive licence) · get_code("a726d4c6815d8975")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned RegressionAlgoConfig. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import warnings
from dataclasses import dataclass, fields
from typing import Any, Mapping, Type, TypeVar

T = TypeVar('T')

def _coerce_dataclass(value: Any, cls: Type[T], *, warning_name: str, default: T) -> T:
    if value is None:
        return default
    if isinstance(value, cls):
        return value
    if isinstance(value, Mapping):
        warnings.warn(f'{warning_name} dictionaries remain supported for backward compatibility, but passing {cls.__name__} is preferred.', DeprecationWarning, stacklevel=3)
        allowed = {field.name for field in fields(cls)}
        filtered = {key: val for key, val in value.items() if key in allowed}
        return cls(**filtered)
    raise TypeError(f'{warning_name} must be a {cls.__name__}, mapping, or None.')

@dataclass(frozen=True)
class RegressionAlgoConfig:
    num_bin_predictor: int = 100
    num_bin_y: int = 80
    binning_method: str = 'quantile'
    interpolation_grid_size: int = 1000
    smooth_kind: str = 'linear'

    def validate(self) -> None:
        if self.num_bin_predictor < 2:
            raise ValueError('num_bin_predictor must be at least 2.')
        if self.num_bin_y < 2:
            raise ValueError('num_bin_y must be at least 2.')
        if self.interpolation_grid_size < 10:
            raise ValueError('interpolation_grid_size must be at least 10.')
        if self.binning_method not in {'quantile', 'fixed'}:
            raise ValueError("binning_method must be 'quantile' or 'fixed'.")
        if self.smooth_kind not in {'linear', 'nearest'}:
            raise ValueError("smooth_kind must be 'linear' or 'nearest'.")

def coerce_regression_algo_config(value: Any) -> RegressionAlgoConfig:
    config = _coerce_dataclass(value, RegressionAlgoConfig, warning_name='algo_params', default=RegressionAlgoConfig())
    config.validate()
    return config

ARGS = {
    'value': None
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type RegressionAlgoConfig — a RegressionAlgoConfig output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a RegressionAlgoConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
coverage_summary Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("306f01011f5f8c9e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# coverage_summary takes three parameters:
# 1. intervals: np.ndarray - should be 2D array with shape (n, 2) containing lower and upper bounds
# 2. y_true: np.ndarray - 1D array of true values to check coverage
# 3. boolean: np.ndarray | None - optional boolean mask for filtering

# Create sample data
intervals = np.array([
    [0.5, 2.5],
    [1.0, 3.0],
], dtype=float)

y_true = np.array([1.0, 2.5], dtype=float)

boolean = np.array([True, True], dtype=bool)

ARGS = {
    "intervals": intervals,
    "y_true": y_true,
    "boolean": boolean
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ensure_1d_array Ran
ran, honoured the contract we drafted
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("3405467e66855b10")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family float, shape (4,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

ARGS = {
    "values": [1.5, 2.3, 3.7, 4.2]
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ndarray, ndim 1, dtype family float, shape (4,) — shape (4,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
linear_extrapolate Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("5c3bd6afe5c56861")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family float, shape (4,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# The function linear_extrapolate takes:
# - x_grid: array-like of x coordinates
# - y_grid: array-like of y values
# - x_new: array-like of new x points to interpolate/extrapolate at
# - kind: string specifying interpolation method ('nearest' or 'linear')

# Create sample data for interpolation
x_grid_values = np.array([1.0, 2.0, 3.0, 4.0, 5.0])
y_grid_values = np.array([2.0, 4.0, 6.0, 8.0, 10.0])
x_new_values = np.array([0.5, 1.5, 3.5, 6.0])

ARGS = {
    'x_grid': x_grid_values,
    'y_grid': y_grid_values,
    'x_new': x_new_values,
    'kind': 'linear'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ndarray, ndim 1, dtype family float, shape (4,) — shape (4,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
make_dense_grid Ran
ran; our contract draft was wrong, not the code
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("8c56d595ddcdc2b1")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family float, shape (50,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# Test with various input types to ensure robustness
values_list = [1.5, 2.3, 0.8, 3.1, 2.0]
grid_size_value = 50
range_padding_value = 0.1

ARGS = {
    "values": values_list,
    "grid_size": grid_size_value,
    "range_padding": range_padding_value,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ndarray, ndim 1, dtype family float, shape (50,) — shape (50,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
make_grid Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("54ea0e1ebe5caf33")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family float, shape (8,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

ARGS = {
    'values': np.array([1.2, 2.5, 3.1, 4.8, 5.3, 6.7, 7.2, 8.9]),
    'num_bin': 5,
    'values_to_add': np.array([0.5, 9.5]),
    'binning_method': 'quantile'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ndarray, ndim 1, dtype family float, shape (8,) — shape (8,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
match_grid_value Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("77b710ef3aad7921")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family float, shape (3,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# Test case 1: Basic usage with simple arrays
values_1 = [1.5, 2.7, 3.2]
grid_1 = np.array([0.0, 1.0, 2.0, 3.0, 4.0])
return_index_1 = False
all_inside_1 = False

# Test case 2: With return_index=True
values_2 = [0.5, 1.5, 2.5]
grid_2 = np.array([0.0, 1.0, 2.0, 3.0])
return_index_2 = True
all_inside_2 = False

# Test case 3: With all_inside=True
values_3 = [5.0, 10.0]
grid_3 = np.array([0.0, 2.0, 4.0, 6.0])
return_index_3 = False
all_inside_3 = True

# Use the first test case for ARGS
ARGS = {
    'values': [1.5, 2.7, 3.2],
    'grid': np.array([0.0, 1.0, 2.0, 3.0, 4.0]),
    'return_index': False,
    'all_inside': False
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ndarray, ndim 1, dtype family float, shape (3,) — shape (3,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
regression_conformity_score Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("92170b23c414e324")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family float, shape (4,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# Construct valid arguments for regression_conformity_score

y_values = np.array([1.5, 2.3, 3.1, 0.9])
calibrated_prediction = 2.0
original_predictions = np.array([1.4, 2.5, 2.9, 1.1])
scoring_method = 'calibrated'

ARGS = {
    'y_values': y_values,
    'calibrated_prediction': calibrated_prediction,
    'original_predictions': original_predictions,
    'scoring_method': scoring_method
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ndarray, ndim 1, dtype family float, shape (4,) — shape (4,) as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
safe_median Ran
ran, honoured the contract we drafted
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("5828d81b7e215a1a")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np
from typing import Iterable

def ensure_1d_array(values) -> np.ndarray:
    array = np.asarray(values, dtype=float)
    if array.ndim == 0:
        array = array.reshape(1)
    return np.ravel(array)

def safe_median(values: Iterable[float]) -> float:
    array = ensure_1d_array(list(values))
    if array.size == 0:
        return 0.0
    return float(np.median(array))

# Test cases for the values parameter
# Case 1: list of floats
values_list = [1.5, 2.7, 3.2, 4.1, 5.9]

# Case 2: numpy array
values_array = np.array([10.1, 20.3, 30.5])

# Case 3: empty iterable
values_empty = []

# Case 4: single element
values_single = [42.0]

# Case 5: tuple
values_tuple = (7.2, 8.4, 9.6, 10.1)

# Main test: use a non-empty list
ARGS = {
    "values": [1.5, 2.7, 3.2, 4.1, 5.9]
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
PredictorProtocol Not yet run
unverified (harvested, no recorded run)
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("3153947df05da460")
How it ran

no invocation recorded.

SelfCalibratingConformalPredictor Not yet run
unverified (harvested, no recorded run)
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py
code served (permissive licence) · get_code("a6e4d4073c18d9c3")
How it ran

no invocation recorded.

Also present in other papers' code

Of 25 samples lifted from this paper's linked repositories, 25 carry a function identity and 0 do not. Of the 24 distinct functions with an identity, 0 are also present in other papers whose code we hold (0 with at least one paper dated earlier, 0 the same month, 0 later) and 24 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

In machine learning, model calibration and predictive inference are essential for producing reliable predictions and quantifying uncertainty to support decisionmaking. Recognizing the complementary roles of point and interval predictions, we introduce Self-Calibrating Conformal Prediction, a method that combines Venn-Abers calibration and conformal prediction to deliver calibrated point predictions alongside prediction intervals with finite-sample validity conditional on these predictions. To achieve this, we extend the original Venn-Abers procedure from binary classification to regression. Our theoretical framework supports analyzing conformal prediction methods that involve calibrating model predictions and subsequently constructing conditionally valid prediction intervals on the same data, where the conditioning set or conformity scores may depend on the calibrated predictions. Real-data experiments show that our method improves interval efficiency through model calibration and offers a practical alternative to feature-conditional validity.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2402.07307")
get_code_for_paper("2402.07307")
have("2402.07307")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2402.07307.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2402.07307.svg)](https://syntology.ai/paper/2402.07307)

Connect an agent — have() is free.