| HistogramConfig |
Ran ran on a synthesized input · constructed an object; no result was computed |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("f2d9b65a76a0a98d") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, HistogramConfig: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
from dataclasses import dataclass
torch.manual_seed(0)
@dataclass(frozen=True)
class HistogramConfig:
num_bin: int = 5
binning_method: str = 'quantile'
def validate(self) -> None:
if self.num_bin < 2:
raise ValueError('num_bin must be at least 2.')
if self.binning_method not in {'quantile', 'fixed'}:
raise ValueError("binning_method must be 'quantile' or 'fixed'.")
ARGS = {
'num_bin': 5,
'binning_method': 'quantile',
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type HistogramConfig — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a HistogramConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| PredictionAdapter |
Ran ran on a synthesized input · constructed an object; no result was computed |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("d0a618b0a8c89ef2") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, PredictionAdapter: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
from dataclasses import dataclass
from typing import Any, Protocol, runtime_checkable
@runtime_checkable
class PredictorProtocol(Protocol):
def predict(self, x: Any) -> Any:
...
def ensure_1d_array(values) -> np.ndarray:
array = np.asarray(values, dtype=float)
if array.ndim == 0:
array = array.reshape(1)
return np.ravel(array)
@dataclass(frozen=True)
class PredictionAdapter:
predictor: Any
def __call__(self, x: Any) -> np.ndarray:
if callable(self.predictor) and (not hasattr(self.predictor, 'predict')):
values = self.predictor(x)
elif isinstance(self.predictor, PredictorProtocol):
values = self.predictor.predict(x)
elif hasattr(self.predictor, 'predict'):
values = self.predictor.predict(x)
else:
raise TypeError('Predictor must be a callable or expose a .predict method.')
return ensure_1d_array(values)
# Test with a simple callable predictor
def simple_predictor(x):
return np.array([1.0, 2.0, 3.0])
# Test with a class that has a predict method
class ModelPredictor:
def predict(self, x: Any) -> Any:
return np.array([4.0, 5.0])
# Test with a callable class (implements __call__)
class CallablePredictor:
def __call__(self, x: Any) -> Any:
return np.array([6.0, 7.0, 8.0, 9.0])
ARGS = {
"predictor": simple_predictor
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type PredictionAdapter — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a PredictionAdapter output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| RegressionAlgoConfig |
Ran ran on a synthesized input · constructed an object; no result was computed |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("9b87b253916c3a4f") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, RegressionAlgoConfig: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
from dataclasses import dataclass
torch.manual_seed(0)
ARGS = {
'num_bin_predictor': 100,
'num_bin_y': 80,
'binning_method': 'quantile',
'interpolation_grid_size': 1000,
'smooth_kind': 'linear',
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type RegressionAlgoConfig — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a RegressionAlgoConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| RegressionFitState |
Ran ran on a synthesized input · constructed an object; no result was computed |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("b8c92398dd17d5da") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, RegressionFitState: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
from dataclasses import dataclass
import pandas as pd
@dataclass
class RegressionFitState:
prediction_grid: np.ndarray
calibrated_prediction: np.ndarray
venn_abers_bounds: np.ndarray
interval_bounds: np.ndarray
raw_venn_paths: np.ndarray
def to_frame(self) -> pd.DataFrame:
return pd.DataFrame({'prediction_uncal': list(self.prediction_grid), 'prediction_cal': list(self.calibrated_prediction), 'prediction_venn_abers': list(self.venn_abers_bounds), 'prediction_interval': list(self.interval_bounds)})
ARGS = {
'prediction_grid': np.array([1.2, 2.3, 3.4, 4.5]),
'calibrated_prediction': np.array([1.1, 2.2, 3.3, 4.4]),
'venn_abers_bounds': np.array([[1.0, 1.3], [2.0, 2.5], [3.0, 3.6], [4.0, 4.7]]),
'interval_bounds': np.array([[0.9, 1.4], [1.9, 2.6], [2.9, 3.7], [3.9, 4.8]]),
'raw_venn_paths': np.array([[0.1, 0.2, 0.3], [0.15, 0.25, 0.35], [0.12, 0.22, 0.32], [0.18, 0.28, 0.38]])
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, pandas — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type RegressionFitState — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a RegressionFitState output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| _coerce_dataclass |
Ran ran; our contract draft was wrong, not the code |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("70511a2064961b91") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned SampleConfig. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import warnings
from dataclasses import dataclass, fields
from typing import Any, Mapping, Type, TypeVar
T = TypeVar('T')
def _coerce_dataclass(value: Any, cls: Type[T], *, warning_name: str, default: T) -> T:
if value is None:
return default
if isinstance(value, cls):
return value
if isinstance(value, Mapping):
warnings.warn(f'{warning_name} dictionaries remain supported for backward compatibility, but passing {cls.__name__} is preferred.', DeprecationWarning, stacklevel=3)
allowed = {field.name for field in fields(cls)}
filtered = {key: val for key, val in value.items() if key in allowed}
return cls(**filtered)
raise TypeError(f'{warning_name} must be a {cls.__name__}, mapping, or None.')
@dataclass
class SampleConfig:
learning_rate: float = 0.001
batch_size: int = 32
hidden_dim: int = 128
ARGS = {
'value': {
'learning_rate': 0.002,
'batch_size': 64,
'hidden_dim': 256
},
'cls': SampleConfig,
'warning_name': 'model_config',
'default': SampleConfig(learning_rate=0.001, batch_size=32, hidden_dim=128)
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type SampleConfig — a SampleConfig output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a SampleConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| _isotonic_block_fit |
Ran ran; our contract draft was wrong, not the code |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("c4b6224294c04c80") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned function. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Define a reducer function that will be passed as the 'reducer' parameter
# The function uses: reducer(np.array(...)) - it calls reducer with a numpy array
def mean_reducer(values):
"""Simple reducer that computes the mean of values"""
return np.mean(values)
# Create sample data for x and y
x_data = np.array([1.0, 2.0, 3.0, 4.0])
y_data = np.array([2.0, 1.0, 3.0, 2.5])
ARGS = {
'x': x_data,
'y': y_data,
'reducer': mean_reducer
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type function — a function output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a function output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| _step_transform |
Ran ran; our contract draft was wrong, not the code |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("ffc7bda663953207") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned function. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Create a small grid of evenly spaced values
grid = np.array([0.0, 1.0, 2.0, 3.0, 4.0])
# Create corresponding values for each grid point
values = np.array([10.0, 20.0, 30.0, 40.0, 50.0])
ARGS = {
"grid": grid,
"values": values,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type function — a function output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a function output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| _weighted_quantile |
Ran ran; our contract draft was wrong, not the code |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("541c619537506f29") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Construct test arguments for _weighted_quantile function
values = np.array([1.5, 2.3, 3.7, 4.2, 5.1, 6.8])
q = 0.5
ARGS = {
"values": values,
"q": q,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| adapt_predictor |
Ran ran; our contract draft was wrong, not the code |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("628015812b19addd") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned PredictionAdapter. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
from dataclasses import dataclass
from typing import Any, Callable, Protocol, runtime_checkable
@runtime_checkable
class PredictorProtocol(Protocol):
def predict(self, x: Any) -> Any:
...
def ensure_1d_array(values) -> np.ndarray:
array = np.asarray(values, dtype=float)
if array.ndim == 0:
array = array.reshape(1)
return np.ravel(array)
@dataclass(frozen=True)
class PredictionAdapter:
predictor: Any
def __call__(self, x: Any) -> np.ndarray:
if callable(self.predictor) and (not hasattr(self.predictor, 'predict')):
values = self.predictor(x)
elif isinstance(self.predictor, PredictorProtocol):
values = self.predictor.predict(x)
elif hasattr(self.predictor, 'predict'):
values = self.predictor.predict(x)
else:
raise TypeError('Predictor must be a callable or expose a .predict method.')
return ensure_1d_array(values)
def adapt_predictor(predictor: Any) -> PredictionAdapter:
return PredictionAdapter(predictor=predictor)
class SimpleCallablePredictor:
"""A simple callable predictor without a predict method."""
def __call__(self, x):
return np.array([1.0, 2.0, 3.0])
class SimplePredictorWithMethod:
"""A predictor with a predict method."""
def predict(self, x):
return np.array([4.0, 5.0])
def simple_function_predictor(x):
"""A simple function that acts as a predictor."""
return np.array([6.0, 7.0, 8.0, 9.0])
ARGS = {
"predictor": SimpleCallablePredictor()
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type PredictionAdapter — a PredictionAdapter output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a PredictionAdapter output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| calibrator_histogram |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("5ed9f5202825f1ce") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned function. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Set up random seed for reproducibility
np.random.seed(0)
# Create synthetic prediction scores and binary labels
# f: model predictions (continuous values between 0 and 1)
# y: ground truth labels (binary: 0 or 1)
f = np.array([0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.85, 0.9], dtype=float)
y = np.array([0, 0, 0, 0, 1, 1, 1, 1, 1, 1], dtype=float)
ARGS = {
'f': f,
'y': y,
'num_bin': 5,
'binning_method': 'quantile',
'quantile_level': None
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type function — a function output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a function output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| calibrator_isotonic |
Ran ran; our contract draft was wrong, not the code |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("9138963f8018cfac") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned function. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
def ensure_1d_array(values) -> np.ndarray:
array = np.asarray(values, dtype=float)
if array.ndim == 0:
array = array.reshape(1)
return np.ravel(array)
def _isotonic_block_fit(x: np.ndarray, y: np.ndarray, reducer):
order = np.argsort(x, kind='mergesort')
x_sorted = ensure_1d_array(x)[order]
y_sorted = ensure_1d_array(y)[order]
blocks: list[dict[str, np.ndarray | float]] = []
for x_value, y_value in zip(x_sorted, y_sorted):
blocks.append({'x_left': float(x_value), 'x_right': float(x_value), 'values': np.array([float(y_value)]), 'prediction': float(reducer(np.array([float(y_value)])))})
while len(blocks) >= 2 and blocks[-2]['prediction'] > blocks[-1]['prediction']:
right = blocks.pop()
left = blocks.pop()
merged_values = np.concatenate([left['values'], right['values']])
blocks.append({'x_left': left['x_left'], 'x_right': right['x_right'], 'values': merged_values, 'prediction': float(reducer(merged_values))})
intervals = [(float(block['x_left']), float(block['x_right']), float(block['prediction'])) for block in blocks]
def transform(x_new):
x_new = ensure_1d_array(x_new)
result = np.empty_like(x_new, dtype=float)
for index, value in enumerate(x_new):
assigned = False
for left, right, prediction in intervals:
if left <= value <= right:
result[index] = prediction
assigned = True
break
if assigned:
continue
if value < intervals[0][0]:
result[index] = intervals[0][2]
else:
result[index] = intervals[-1][2]
return result
return transform
def calibrator_isotonic(f: np.ndarray, y: np.ndarray, **kwargs):
del kwargs
return _isotonic_block_fit(f, y, reducer=np.mean)
f_values = np.array([0.1, 0.3, 0.2, 0.5, 0.4, 0.6], dtype=float)
y_values = np.array([0.0, 1.0, 0.0, 1.0, 1.0, 1.0], dtype=float)
ARGS = {
'f': f_values,
'y': y_values,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type function — a function output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a function output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| coerce_histogram_config |
Ran ran; our contract draft was wrong, not the code |
Larsvanderlaan/SelfCalibratingConformal/selfcalibratingconformal/configs.py code served (permissive licence) · get_code("a5218d94fbe5bb1e") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned HistogramConfig. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import warnings
from dataclasses import dataclass, fields
from typing import Any, Mapping, Type, TypeVar
T = TypeVar('T')
def _coerce_dataclass(value: Any, cls: Type[T], *, warning_name: str, default: T) -> T:
if value is None:
return default
if isinstance(value, cls):
return value
if isinstance(value, Mapping):
warnings.warn(f'{warning_name} dictionaries remain supported for backward compatibility, but passing {cls.__name__} is preferred.', DeprecationWarning, stacklevel=3)
allowed = {field.name for field in fields(cls)}
filtered = {key: val for key, val in value.items() if key in allowed}
return cls(**filtered)
raise TypeError(f'{warning_name} must be a {cls.__name__}, mapping, or None.')
@dataclass(frozen=True)
class HistogramConfig:
num_bin: int = 5
binning_method: str = 'quantile'
def validate(self) -> None:
if self.num_bin < 2:
raise ValueError('num_bin must be at least 2.')
if self.binning_method not in {'quantile', 'fixed'}:
raise ValueError("binning_method must be 'quantile' or 'fixed'.")
def coerce_histogram_config(value: Any, *, warning_name: str) -> HistogramConfig:
config = _coerce_dataclass(value, HistogramConfig, warning_name=warning_name, default=HistogramConfig())
config.validate()
return config
ARGS = {
'value': {
'num_bin': 10,
'binning_method': 'quantile'
},
'warning_name': 'test_histogram_config'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type HistogramConfig — a HistogramConfig output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a HistogramConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| coerce_quantile_algo_config |
Ran ran; our contract draft was wrong, not the code |
Larsvanderlaan/SelfCalibratingConformal/selfcalibratingconformal/configs.py code served (permissive licence) · get_code("d07d076cff2b6832") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned QuantileAlgoConfig. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import warnings
from dataclasses import dataclass, fields
from typing import Any, Mapping, Type, TypeVar
T = TypeVar('T')
def _coerce_dataclass(value: Any, cls: Type[T], *, warning_name: str, default: T) -> T:
if value is None:
return default
if isinstance(value, cls):
return value
if isinstance(value, Mapping):
warnings.warn(f'{warning_name} dictionaries remain supported for backward compatibility, but passing {cls.__name__} is preferred.', DeprecationWarning, stacklevel=3)
allowed = {field.name for field in fields(cls)}
filtered = {key: val for key, val in value.items() if key in allowed}
return cls(**filtered)
raise TypeError(f'{warning_name} must be a {cls.__name__}, mapping, or None.')
@dataclass(frozen=True)
class QuantileAlgoConfig:
num_bin_predictor: int = 100
num_bin_score: int = 120
binning_method: str = 'quantile'
interpolation_grid_size: int = 800
score_range_padding: float = 0.05
def validate(self) -> None:
if self.num_bin_predictor < 2:
raise ValueError('num_bin_predictor must be at least 2.')
if self.num_bin_score < 10:
raise ValueError('num_bin_score must be at least 10.')
if self.interpolation_grid_size < 50:
raise ValueError('interpolation_grid_size must be at least 50.')
if self.binning_method not in {'quantile', 'fixed'}:
raise ValueError("binning_method must be 'quantile' or 'fixed'.")
if self.score_range_padding < 0:
raise ValueError('score_range_padding must be non-negative.')
def coerce_quantile_algo_config(value: Any) -> QuantileAlgoConfig:
config = _coerce_dataclass(value, QuantileAlgoConfig, warning_name='algo_params', default=QuantileAlgoConfig())
config.validate()
return config
ARGS = {
'value': {
'num_bin_predictor': 50,
'num_bin_score': 100,
'binning_method': 'quantile',
'interpolation_grid_size': 500,
'score_range_padding': 0.1
}
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type QuantileAlgoConfig — a QuantileAlgoConfig output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a QuantileAlgoConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| coerce_regression_algo_config |
Ran ran; our contract draft was wrong, not the code |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("143bbabb14851e55") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned RegressionAlgoConfig. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import numpy as np
from typing import Any, Mapping
torch.manual_seed(0)
# Test case 1: None value (should use default)
ARGS = {
'value': None
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type RegressionAlgoConfig — a RegressionAlgoConfig output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a RegressionAlgoConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| coerce_regression_algo_config |
Ran ran; our contract draft was wrong, not the code |
Larsvanderlaan/SelfCalibratingConformal/selfcalibratingconformal/configs.py code served (permissive licence) · get_code("a726d4c6815d8975") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned RegressionAlgoConfig. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import warnings
from dataclasses import dataclass, fields
from typing import Any, Mapping, Type, TypeVar
T = TypeVar('T')
def _coerce_dataclass(value: Any, cls: Type[T], *, warning_name: str, default: T) -> T:
if value is None:
return default
if isinstance(value, cls):
return value
if isinstance(value, Mapping):
warnings.warn(f'{warning_name} dictionaries remain supported for backward compatibility, but passing {cls.__name__} is preferred.', DeprecationWarning, stacklevel=3)
allowed = {field.name for field in fields(cls)}
filtered = {key: val for key, val in value.items() if key in allowed}
return cls(**filtered)
raise TypeError(f'{warning_name} must be a {cls.__name__}, mapping, or None.')
@dataclass(frozen=True)
class RegressionAlgoConfig:
num_bin_predictor: int = 100
num_bin_y: int = 80
binning_method: str = 'quantile'
interpolation_grid_size: int = 1000
smooth_kind: str = 'linear'
def validate(self) -> None:
if self.num_bin_predictor < 2:
raise ValueError('num_bin_predictor must be at least 2.')
if self.num_bin_y < 2:
raise ValueError('num_bin_y must be at least 2.')
if self.interpolation_grid_size < 10:
raise ValueError('interpolation_grid_size must be at least 10.')
if self.binning_method not in {'quantile', 'fixed'}:
raise ValueError("binning_method must be 'quantile' or 'fixed'.")
if self.smooth_kind not in {'linear', 'nearest'}:
raise ValueError("smooth_kind must be 'linear' or 'nearest'.")
def coerce_regression_algo_config(value: Any) -> RegressionAlgoConfig:
config = _coerce_dataclass(value, RegressionAlgoConfig, warning_name='algo_params', default=RegressionAlgoConfig())
config.validate()
return config
ARGS = {
'value': None
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type RegressionAlgoConfig — a RegressionAlgoConfig output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a RegressionAlgoConfig output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| coverage_summary |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("306f01011f5f8c9e") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned list, ndim 1, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# coverage_summary takes three parameters:
# 1. intervals: np.ndarray - should be 2D array with shape (n, 2) containing lower and upper bounds
# 2. y_true: np.ndarray - 1D array of true values to check coverage
# 3. boolean: np.ndarray | None - optional boolean mask for filtering
# Create sample data
intervals = np.array([
[0.5, 2.5],
[1.0, 3.0],
], dtype=float)
y_true = np.array([1.0, 2.5], dtype=float)
boolean = np.array([True, True], dtype=bool)
ARGS = {
"intervals": intervals,
"y_true": y_true,
"boolean": boolean
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type list, ndim 1, dtype family float — a list output has no array shape of its own; none is stated
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| ensure_1d_array |
Ran ran, honoured the contract we drafted |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("3405467e66855b10") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family float, shape (4,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
ARGS = {
"values": [1.5, 2.3, 3.7, 4.2]
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ndarray, ndim 1, dtype family float, shape (4,) — shape (4,) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| linear_extrapolate |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("5c3bd6afe5c56861") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family float, shape (4,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# The function linear_extrapolate takes:
# - x_grid: array-like of x coordinates
# - y_grid: array-like of y values
# - x_new: array-like of new x points to interpolate/extrapolate at
# - kind: string specifying interpolation method ('nearest' or 'linear')
# Create sample data for interpolation
x_grid_values = np.array([1.0, 2.0, 3.0, 4.0, 5.0])
y_grid_values = np.array([2.0, 4.0, 6.0, 8.0, 10.0])
x_new_values = np.array([0.5, 1.5, 3.5, 6.0])
ARGS = {
'x_grid': x_grid_values,
'y_grid': y_grid_values,
'x_new': x_new_values,
'kind': 'linear'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ndarray, ndim 1, dtype family float, shape (4,) — shape (4,) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| make_dense_grid |
Ran ran; our contract draft was wrong, not the code |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("8c56d595ddcdc2b1") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family float, shape (50,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Test with various input types to ensure robustness
values_list = [1.5, 2.3, 0.8, 3.1, 2.0]
grid_size_value = 50
range_padding_value = 0.1
ARGS = {
"values": values_list,
"grid_size": grid_size_value,
"range_padding": range_padding_value,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ndarray, ndim 1, dtype family float, shape (50,) — shape (50,) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| make_grid |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("54ea0e1ebe5caf33") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family float, shape (8,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
ARGS = {
'values': np.array([1.2, 2.5, 3.1, 4.8, 5.3, 6.7, 7.2, 8.9]),
'num_bin': 5,
'values_to_add': np.array([0.5, 9.5]),
'binning_method': 'quantile'
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ndarray, ndim 1, dtype family float, shape (8,) — shape (8,) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| match_grid_value |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("77b710ef3aad7921") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family float, shape (3,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Test case 1: Basic usage with simple arrays
values_1 = [1.5, 2.7, 3.2]
grid_1 = np.array([0.0, 1.0, 2.0, 3.0, 4.0])
return_index_1 = False
all_inside_1 = False
# Test case 2: With return_index=True
values_2 = [0.5, 1.5, 2.5]
grid_2 = np.array([0.0, 1.0, 2.0, 3.0])
return_index_2 = True
all_inside_2 = False
# Test case 3: With all_inside=True
values_3 = [5.0, 10.0]
grid_3 = np.array([0.0, 2.0, 4.0, 6.0])
return_index_3 = False
all_inside_3 = True
# Use the first test case for ARGS
ARGS = {
'values': [1.5, 2.7, 3.2],
'grid': np.array([0.0, 1.0, 2.0, 3.0, 4.0]),
'return_index': False,
'all_inside': False
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ndarray, ndim 1, dtype family float, shape (3,) — shape (3,) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| regression_conformity_score |
Ran ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT) |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("92170b23c414e324") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned ndarray, ndim 1, dtype family float, shape (4,) as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Construct valid arguments for regression_conformity_score
y_values = np.array([1.5, 2.3, 3.1, 0.9])
calibrated_prediction = 2.0
original_predictions = np.array([1.4, 2.5, 2.9, 1.1])
scoring_method = 'calibrated'
ARGS = {
'y_values': y_values,
'calibrated_prediction': calibrated_prediction,
'original_predictions': original_predictions,
'scoring_method': scoring_method
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type ndarray, ndim 1, dtype family float, shape (4,) — shape (4,) as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| safe_median |
Ran ran, honoured the contract we drafted |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("5828d81b7e215a1a") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
from typing import Iterable
def ensure_1d_array(values) -> np.ndarray:
array = np.asarray(values, dtype=float)
if array.ndim == 0:
array = array.reshape(1)
return np.ravel(array)
def safe_median(values: Iterable[float]) -> float:
array = ensure_1d_array(list(values))
if array.size == 0:
return 0.0
return float(np.median(array))
# Test cases for the values parameter
# Case 1: list of floats
values_list = [1.5, 2.7, 3.2, 4.1, 5.9]
# Case 2: numpy array
values_array = np.array([10.1, 20.3, 30.5])
# Case 3: empty iterable
values_empty = []
# Case 4: single element
values_single = [42.0]
# Case 5: tuple
values_tuple = (7.2, 8.4, 9.6, 10.1)
# Main test: use a non-empty list
ARGS = {
"values": [1.5, 2.7, 3.2, 4.1, 5.9]
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| PredictorProtocol |
Not yet run unverified (harvested, no recorded run) |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("3153947df05da460") How it ran |
| SelfCalibratingConformalPredictor |
Not yet run unverified (harvested, no recorded run) |
larsvanderlaan/selfcalibratingconformal/selfcalibratingconformal/regression.py code served (permissive licence) · get_code("a6e4d4073c18d9c3") How it ran |