| crps_quantile |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/metrics/crps.py code served (permissive licence) · get_code("653bfa7827b0a9c9") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float64, ndim 0, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Analyze the function signature and body:
# - target: np.array with shape (batch_size, spatial_dim)
# - samples: np.array with shape (num_samples, batch_size, spatial_dim)
# - quantiles: np.array of quantile values (default provided)
# From assertions:
# target.shape[0] == samples.shape[1] (batch dimension matches)
# target.shape[1] == samples.shape[2] (spatial dimension matches)
# Set up small distinct dimensions
batch_size = 2
spatial_dim = 4
num_samples = 5
# Create target array with shape (batch_size, spatial_dim)
target = np.random.randn(batch_size, spatial_dim).astype(np.float32)
# Create samples array with shape (num_samples, batch_size, spatial_dim)
samples = np.random.randn(num_samples, batch_size, spatial_dim).astype(np.float32)
# Create quantiles array (can use default or custom)
quantiles = (np.arange(19) / 19.0)[1:] # excludes 0, includes values up to ~1.0
ARGS = {
"target": target,
"samples": samples,
"quantiles": quantiles,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float64, ndim 0, dtype family float — no output shape is stated: observe_signature.py recorded float ndim 0, which disagrees with the fixture run
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| crps_slow |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/metrics/crps.py code served (permissive licence) · get_code("683b8dfe9350f6f8") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float32, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Construct valid arguments for crps_slow function
# The function expects:
# - target: np.array with shape (batch_size, dim)
# - samples: np.array with shape (num_samples, batch_size, dim)
batch_size = 4
dim = 3
num_samples = 5
# Create target array with shape (batch_size, dim)
target = np.random.randn(batch_size, dim).astype(np.float32)
# Create samples array with shape (num_samples, batch_size, dim)
samples = np.random.randn(num_samples, batch_size, dim).astype(np.float32)
ARGS = {
"target": target,
"samples": samples,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float32, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| dawid_sebastiani_score |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/metrics/dawid_sebastiani.py code served (permissive licence) · get_code("58cab579ed6bb634") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float64, ndim 0, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Set random seed for reproducibility
np.random.seed(42)
# Create small valid arrays
# target shape: (num_features_0, num_features_1) = (4, 3)
# samples shape: (num_samples, num_features_0, num_features_1) = (2, 4, 3)
target = np.random.randn(4, 3).astype(np.float32)
samples = np.random.randn(2, 4, 3).astype(np.float32)
ARGS = {
"target": target,
"samples": samples,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float64, ndim 0, dtype family float — no output shape is stated: observe_signature.py recorded float ndim 0, which disagrees with the fixture run
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| energy_score |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/metrics/energy.py code served (permissive licence) · get_code("ec34b47822b2cf32") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float32, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Set random seed for reproducibility
np.random.seed(0)
# Create small test arrays
# target shape: (height, width) = (3, 4)
target = np.random.randn(3, 4).astype(np.float32)
# samples shape: (num_samples, height, width) = (2, 3, 4)
samples = np.random.randn(2, 3, 4).astype(np.float32)
# beta parameter
beta = 1.0
ARGS = {
"target": target,
"samples": samples,
"beta": beta,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float32, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| energy_score_fast |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/metrics/energy.py code served (permissive licence) · get_code("086d8758cc92c2f9") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float32, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Construct arguments for energy_score_fast
# target: np.array with shape (d1, d2)
# samples: np.array with shape (num_samples, d1, d2)
# beta: float between 0 and 2
# Using small dimensions: d1=4, d2=3, num_samples=4
d1 = 4
d2 = 3
num_samples = 4
# Create target array of shape (d1, d2)
target = np.random.randn(d1, d2).astype(np.float32)
# Create samples array of shape (num_samples, d1, d2)
samples = np.random.randn(num_samples, d1, d2).astype(np.float32)
# Create beta value between 0 and 2
beta = 1.0
ARGS = {
"target": target,
"samples": samples,
"beta": beta
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float32, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| exponential_cross_entropy |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/experiments/kl_divergence.py code served (permissive licence) · get_code("622a08d2aedbaeaa") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float64, ndim 0, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Create small numpy arrays for the function parameters
scale1 = np.array([1.0, 2.0, 3.0, 0.5])
scale2 = np.array([2.0, 1.5, 4.0, 1.0])
ARGS = {
"scale1": scale1,
"scale2": scale2,
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float64, ndim 0, dtype family float — no output shape is stated: observe_signature.py recorded float ndim 0, which disagrees with the fixture run
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| gaussian_cross_entropy |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/experiments/kl_divergence.py code served (permissive licence) · get_code("42eb1613132e5bd3") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float64, ndim 0, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Create small covariance matrices and mean vectors
# Using distinct dimensions: 4 for the dimension of the Gaussian
np.random.seed(0)
# Mean vectors (dimension 4)
mu1 = np.array([0.1, 0.2, 0.3, 0.4])
mu2 = np.array([0.5, 0.6, 0.7, 0.8])
# Covariance matrices (4x4, positive definite)
# Create positive definite matrices using A @ A.T
A1 = np.random.randn(4, 4)
cov1 = A1 @ A1.T
A2 = np.random.randn(4, 4)
cov2 = A2 @ A2.T
ARGS = {
"mu1": mu1,
"cov1": cov1,
"mu2": mu2,
"cov2": cov2
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float64, ndim 0, dtype family float — no output shape is stated: observe_signature.py recorded float ndim 0, which disagrees with the fixture run
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| gaussian_kl_divergence |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/experiments/kl_divergence.py code served (permissive licence) · get_code("5b1713c369009179") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float64, ndim 0, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Set random seed for reproducibility
np.random.seed(0)
# Define small Gaussian parameters
# mu: mean vector (dimension 4)
# cov: covariance matrix (4x4, symmetric positive definite)
dim = 4
# First Gaussian (ground truth)
mu_gt = np.array([0.0, 0.1, 0.2, 0.3])
cov_gt = np.array([
[1.0, 0.1, 0.0, 0.0],
[0.1, 1.2, 0.1, 0.0],
[0.0, 0.1, 1.1, 0.05],
[0.0, 0.0, 0.05, 1.0]
])
# Second Gaussian (forecast)
mu_fcst = np.array([0.05, 0.15, 0.25, 0.35])
cov_fcst = np.array([
[1.1, 0.05, 0.0, 0.0],
[0.05, 1.3, 0.08, 0.0],
[0.0, 0.08, 1.2, 0.03],
[0.0, 0.0, 0.03, 1.05]
])
ARGS = {
"mu_gt": mu_gt,
"cov_gt": cov_gt,
"mu_fcst": mu_fcst,
"cov_fcst": cov_fcst
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float64, ndim 0, dtype family float — no output shape is stated: observe_signature.py recorded float ndim 0, which disagrees with the fixture run
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| gaussian_mixture_wasserstein_distance |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/experiments/wasserstein.py code served (permissive licence) · get_code("e9707f076eb0fd4d") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
from scipy.linalg import sqrtm
# Set random seed for reproducibility
np.random.seed(0)
# Define dimensions
dim = 4
n_components_1 = 2
n_components_2 = 3
# Create weights for first Gaussian mixture
weights1 = [0.3, 0.7]
# Create weights for second Gaussian mixture
weights2 = [0.2, 0.5, 0.3]
# Create means (mu) for first mixture
mu1 = [
np.random.randn(dim),
np.random.randn(dim)
]
# Create covariances for first mixture (must be positive definite)
cov1 = []
for _ in range(n_components_1):
A = np.random.randn(dim, dim)
cov1.append(A @ A.T + np.eye(dim))
# Create means (mu) for second mixture
mu2 = [
np.random.randn(dim),
np.random.randn(dim),
np.random.randn(dim)
]
# Create covariances for second mixture (must be positive definite)
cov2 = []
for _ in range(n_components_2):
A = np.random.randn(dim, dim)
cov2.append(A @ A.T + np.eye(dim))
ARGS = {
"weights1": weights1,
"weights2": weights2,
"mu1": mu1,
"cov1": cov1,
"mu2": mu2,
"cov2": cov2
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, scipy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| gaussian_wasserstein_distance |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/experiments/wasserstein.py code served (permissive licence) · get_code("86b1b5e8b1f99a49") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float64, ndim 0, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
from scipy.linalg import sqrtm
np.random.seed(0)
# Create two sets of parameters for two Gaussian distributions
# mu1, cov1 for first Gaussian
mu1 = np.array([1.0, 2.0, 3.0, 4.0])
cov1 = np.array([
[2.0, 0.1, 0.0, 0.0],
[0.1, 2.5, 0.1, 0.0],
[0.0, 0.1, 2.0, 0.1],
[0.0, 0.0, 0.1, 2.2]
])
# mu2, cov2 for second Gaussian
mu2 = np.array([1.5, 2.5, 3.5, 4.5])
cov2 = np.array([
[1.8, 0.05, 0.0, 0.0],
[0.05, 2.3, 0.05, 0.0],
[0.0, 0.05, 1.9, 0.05],
[0.0, 0.0, 0.05, 2.1]
])
ARGS = {
"mu1": mu1,
"cov1": cov1,
"mu2": mu2,
"cov2": cov2
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, scipy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float64, ndim 0, dtype family float — no output shape is stated: observe_signature.py recorded float ndim 0, which disagrees with the fixture run
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| kernel_laplace |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/metrics/kernel.py code served (permissive licence) · get_code("a89c62ddaf8564cf") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float32, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Construct valid arguments for kernel_laplace
# The function expects:
# - target: np.array with shape (height, width)
# - samples: np.array with shape (num_samples, height, width)
# - gamma: float (default=1.0)
# Looking at the assertions:
# target.shape[0] == samples.shape[1] -> target height == samples height
# target.shape[1] == samples.shape[2] -> target width == samples width
# Create small test arrays with distinct dimensions for clarity
height = 2
width = 3
num_samples = 4
target = np.random.randn(height, width).astype(np.float32)
samples = np.random.randn(num_samples, height, width).astype(np.float32)
gamma = 1.0
ARGS = {
"target": target,
"samples": samples,
"gamma": gamma
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float32, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| kernel_laplace_spherical |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/metrics/kernel.py code served (permissive licence) · get_code("0d2316045e5123c8") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float32, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Construct valid arguments for kernel_laplace_spherical
# The function expects:
# - target: np.array with shape (d1, d2)
# - samples: np.array with shape (num_samples, d1, d2)
# - gamma: float (default 1.0)
# Looking at the assertions:
# assert target.shape[0] == samples.shape[1]
# assert target.shape[1] == samples.shape[2]
# So if target is shape (4, 3), samples must be shape (num_samples, 4, 3)
np.random.seed(0)
target = np.random.randn(4, 3).astype(np.float32)
samples = np.random.randn(2, 4, 3).astype(np.float32)
gamma = 1.0
ARGS = {
"target": target,
"samples": samples,
"gamma": gamma
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float32, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| kernel_matern |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/metrics/kernel.py code served (permissive licence) · get_code("cf9541f4fe97a349") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float64, ndim 0, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Construct valid arguments for kernel_matern
# The function expects:
# - target: np.array with shape (d1, d2) where d1*d2 = dim
# - samples: np.array with shape (num_samples, d1, d2)
# - nu: float (default 1.5)
# - gamma: float (default 1.0)
# Let's create small, distinct dimensions
# target will be shape (2, 3) -> dim=6
# samples will be shape (4, 2, 3) -> 4 samples of 2x3
target = np.random.randn(2, 3).astype(np.float64)
samples = np.random.randn(4, 2, 3).astype(np.float64)
nu = 1.5
gamma = 1.0
ARGS = {
"target": target,
"samples": samples,
"nu": nu,
"gamma": gamma
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy, sklearn — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float64, ndim 0, dtype family float — no output shape is stated: observe_signature.py recorded float ndim 0, which disagrees with the fixture run
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| select_params |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/experiments/h0_vs_h1.py code served (permissive licence) · get_code("ec4c5581d554fa67") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned function. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import torch
import torch.nn as nn
torch.manual_seed(0)
def dummy_func(target, samples, **kwargs):
"""A simple function that uses target and samples"""
return target + samples.sum()
ARGS = {
'func': dummy_func,
'kwargs': {'multiplier': 2, 'offset': 5}
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- torch — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type function — a function output has no array shape of its own; none is stated — the instrument coerces any output to an array; for a function output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| variogram_score_iter |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/metrics/variogram.py code served (permissive licence) · get_code("7db8a498674767fb") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float32, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Create small test arrays
# target shape: (height, width) - the spatial dimensions
# samples shape: (num_samples, height, width) - ensemble of predictions
height = 3
width = 4
num_samples = 2
# Create target array with shape (3, 4)
target = np.array([
[1.0, 2.0, 3.0, 4.0],
[5.0, 6.0, 7.0, 8.0],
[9.0, 10.0, 11.0, 12.0]
], dtype=np.float32)
# Create samples array with shape (2, 3, 4)
# 2 samples, each with shape (3, 4)
samples = np.array([
[[1.1, 2.1, 3.1, 4.1],
[5.1, 6.1, 7.1, 8.1],
[9.1, 10.1, 11.1, 12.1]],
[[0.9, 1.9, 2.9, 3.9],
[4.9, 5.9, 6.9, 7.9],
[8.9, 9.9, 10.9, 11.9]]
], dtype=np.float32)
p = 1.0
ARGS = {
"target": target,
"samples": samples,
"p": p
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float32, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| variogram_score_mixed |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/metrics/variogram.py code served (permissive licence) · get_code("7a7e624a0059daf1") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float32, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Construct arguments for variogram_score_mixed
# target.shape[0] == samples.shape[1]
# target.shape[1] == samples.shape[2]
# samples.shape[0] is the number of samples (first dimension)
# Let's use:
# target: shape (3, 4) - 3 rows, 4 columns
# samples: shape (2, 3, 4) - 2 samples, 3 rows, 4 columns
target = np.random.randn(3, 4).astype(np.float32)
samples = np.random.randn(2, 3, 4).astype(np.float32)
p = 1.0
ARGS = {
"target": target,
"samples": samples,
"p": p
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float32, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| variogram_score_tensor |
Ran ran on a synthesized input |
ServiceNow/regions-of-reliability/ror/metrics/variogram.py code served (permissive licence) · get_code("b76e853f4dce8b7b") How it ranRan in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned float32, ndim 0, dtype family float, shape () as a later run recorded it. The image it ran in was not recorded. If your copy returns the same type, ndim and shape on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.
import numpy as np
# Construct valid arguments for variogram_score_tensor
# The function expects:
# - target: np.array with shape (H, W)
# - samples: np.array with shape (N, H, W) where N is number of samples
# - p: float > 0
# Create target array with shape (4, 3)
target = np.random.randn(4, 3).astype(np.float32)
# Create samples array with shape (2, 4, 3)
# First dimension is number of samples (2)
# Second and third dimensions match target shape (4, 3)
samples = np.random.randn(2, 4, 3).astype(np.float32)
# Create p parameter
p = 1.0
ARGS = {
"target": target,
"samples": samples,
"p": p
}
- Setup block drafted by
- a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
- Imports
- numpy — import names as written, not package names to install; a name the repository itself provides is not a package
- Returned
- type float32, ndim 0, dtype family float, shape () — shape () as a later run recorded it
- dtype family
- int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
- Environment
- image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
|
| compute_metrics_from_benchmark |
Not yet run unverified (harvested, no recorded run) |
ServiceNow/regions-of-reliability/ror/benchmark_data.py code served (permissive licence) · get_code("571e637ac68c5e1d") How it ran |
| compute_metrics_single_experiment |
Not yet run unverified (harvested, no recorded run) |
ServiceNow/regions-of-reliability/ror/benchmark_data.py code served (permissive licence) · get_code("c764476c7f3110be") How it ran |