SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2210.04642 · NeurIPS · 2022

Exploration via Planning for Information about the Optimal Trajectory

Stefano Ermon, Jeff Schneider, Viraj Mehta, Willie Neiswanger, Ian Char, Joseph Abbate, Rory Conlin, Mark Boyer

arXiv · PDF · Open in the Atlas

Code that ran

6 of 8 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (6 of them constructed an object rather than computing a result; 0 honoured the contract we drafted, 0 violated it and 6 have no contract verdict; 0 failures of our instrument; 2 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2210.04642.json · badge: /api/ran/2210.04642.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
rehoss/lbmpc_semimarkov community
Papers with Code archive
6 ran · 2 no recorded run
FunctionStatusWhere it lives
AcqFunction Ran
ran on a synthesized input · constructed an object; no result was computed
rehoss/lbmpc_semimarkov/lbmpc_semimarkov/acq/acquisition.py
code served (permissive licence) · get_code("80237591e7b4d3eb")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, AcqFunction: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np
import copy
from argparse import Namespace

# Create a minimal mock model that satisfies the requirements
class SimpleGp:
    """Minimal mock GP model."""
    def __init__(self):
        self.name = "SimpleGp"
        self.params = Namespace(verbose=False)
    
    def __deepcopy__(self, memo):
        """Support deepcopy as used in set_model."""
        new_model = SimpleGp()
        memo[id(self)] = new_model
        return new_model

def dict_to_namespace(params):
    """Convert dict to Namespace if needed."""
    if isinstance(params, dict):
        params = Namespace(**params)
    return params

# Create a minimal params dict/Namespace for AcqFunction
params_dict = {
    'name': 'TestAcqFunction',
    'verbose': False,
    'dumper': None
}

# Create a model instance
model_instance = SimpleGp()

# Define all arguments for AcqFunction.__init__
ARGS = {
    'params': params_dict,
    'model': model_instance,
    'verbose': True,
    'kwargs': {}
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type AcqFunction — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a AcqFunction output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
AlgorithmSet Ran
ran on a synthesized input · constructed an object; no result was computed
rehoss/lbmpc_semimarkov/lbmpc_semimarkov/acq/acquisition.py
code served (permissive licence) · get_code("85c1f25773a865b6")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, AlgorithmSet: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

class ExecutionPath:
    """Simple container for execution path data."""
    def __init__(self):
        self.x = []
        self.y = []

class Algorithm:
    """Minimal Algorithm implementation with required methods."""
    def __init__(self):
        self.exe_path = ExecutionPath()
        self.query_count = 0
        self.max_queries = 3
        self.output_value = 42.0
    
    def initialize(self):
        """Initialize the algorithm."""
        self.exe_path = ExecutionPath()
        self.query_count = 0
    
    def get_next_x(self):
        """Return next x to evaluate, or None when done."""
        if self.query_count < self.max_queries:
            self.query_count += 1
            return torch.randn(4)
        return None
    
    def get_copy(self):
        """Return a copy of this algorithm."""
        new_algo = Algorithm()
        new_algo.exe_path = ExecutionPath()
        new_algo.query_count = self.query_count
        new_algo.max_queries = self.max_queries
        new_algo.output_value = self.output_value
        return new_algo
    
    def get_output(self):
        """Return final output."""
        return self.output_value
    
    def get_exe_path_crop(self):
        """Return cropped execution path."""
        cropped_path = ExecutionPath()
        cropped_path.x = self.exe_path.x[:]
        cropped_path.y = self.exe_path.y[:]
        return cropped_path

def f_list_func(x_list):
    """Dummy function list that evaluates all x values."""
    results = []
    for x in x_list:
        if x is not None:
            results.append(torch.sum(x).item())
        else:
            results.append(None)
    return results

algo = Algorithm()

ARGS = {
    'algo': algo
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type AlgorithmSet — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a AlgorithmSet output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Base Ran
ran on a synthesized input · constructed an object; no result was computed
rehoss/lbmpc_semimarkov/lbmpc_semimarkov/acq/acquisition.py
code served (permissive licence) · get_code("fa3292dc4951b1c6")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Base: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from argparse import Namespace

torch.manual_seed(0)

# Examine the Base class to see what parameters are used:
# __init__ takes: params (Namespace or dict), verbose (bool)
# set_params extracts from params: name, verbose, dumper
# The params are used in print_init() and __str__()

# Create a minimal params dict/Namespace with all possible attributes
params_dict = {
    'name': 'TestModel',
    'verbose': True,
    'dumper': None
}

ARGS = {
    'params': params_dict,
    'verbose': True
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Base — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Base output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
BatchAlgorithmSet Ran
ran on a synthesized input · constructed an object; no result was computed
rehoss/lbmpc_semimarkov/lbmpc_semimarkov/acq/acquisition.py
code served (permissive licence) · get_code("abd4eaae6be6d5b6")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, BatchAlgorithmSet: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# First, define the ExecutionPath class that algorithms use
class ExecutionPath:
    def __init__(self):
        self.x = []
        self.y = []

# Define a minimal Algorithm class
class Algorithm:
    def __init__(self, max_steps=3):
        self.max_steps = max_steps
        self.step_count = 0
        self.exe_path = ExecutionPath()
    
    def initialize(self):
        self.step_count = 0
        self.exe_path = ExecutionPath()
    
    def get_next_x(self):
        if self.step_count < self.max_steps:
            self.step_count += 1
            return torch.randn(2, 4)
        return None
    
    def get_copy(self):
        new_algo = Algorithm(max_steps=self.max_steps)
        new_algo.initialize()
        return new_algo
    
    def get_output(self):
        return torch.tensor([1.0, 2.0])
    
    def get_exe_path_crop(self):
        return self.exe_path

# Define a minimal BatchAlgorithm class
class BatchAlgorithm:
    def __init__(self, max_batches=2, batch_size=2):
        self.max_batches = max_batches
        self.batch_size = batch_size
        self.batch_count = 0
        self.exe_path = ExecutionPath()
    
    def initialize(self):
        self.batch_count = 0
        self.exe_path = ExecutionPath()
    
    def get_next_x_batch(self):
        if self.batch_count < self.max_batches:
            self.batch_count += 1
            return [torch.randn(2, 4) for _ in range(self.batch_size)]
        return []
    
    def get_copy(self):
        new_algo = BatchAlgorithm(max_batches=self.max_batches, batch_size=self.batch_size)
        new_algo.initialize()
        return new_algo
    
    def get_output(self):
        return torch.tensor([3.0, 4.0])
    
    def get_exe_path_crop(self):
        return self.exe_path

# Create instances for testing
batch_algo = BatchAlgorithm(max_batches=2, batch_size=2)

# Define f_batch_list - a function that takes a list of x_batch lists and returns y_batch lists
def f_batch_list(x_batch_list):
    """Process each batch in x_batch_list and return corresponding y batches."""
    y_batch_list = []
    for x_batch in x_batch_list:
        if len(x_batch) > 0:
            # Sum across batch dimension for each input
            y_batch = [torch.ones(2, 3) for _ in x_batch]
            y_batch_list.append(y_batch)
        else:
            y_batch_list.append([])
    return y_batch_list

# Define n_f - number of parallel functions
n_f = 3

ARGS = {
    'algo': batch_algo,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type BatchAlgorithmSet — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a BatchAlgorithmSet output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
FunctionSample Ran
ran on a synthesized input · constructed an object; no result was computed
rehoss/lbmpc_semimarkov/lbmpc_semimarkov/acq/acquisition.py
code served (permissive licence) · get_code("095a77eeda8d0fd2")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, FunctionSample: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
from argparse import Namespace
import copy
import logging

torch.manual_seed(0)

# Create a minimal mock model that satisfies the interface used by FunctionSample
class MockModel:
    def __init__(self):
        self.data = Namespace(x=[], y=[])
    
    def set_data(self, data):
        self.data = copy.deepcopy(data)
    
    def get_post_mu_cov(self, x_list, full_cov=False):
        # Return posterior mean and std for each x in x_list
        if isinstance(x_list, list):
            n = len(x_list)
        else:
            n = x_list.shape[0] if hasattr(x_list, 'shape') else 1
        
        mean_arr = np.random.randn(n, 2)
        std_arr = np.abs(np.random.randn(n, 2))
        return (mean_arr, std_arr)
    
    def sample_post_pred(self, x, num_samples):
        # Sample posterior predictive at x
        # Returns array of shape (num_samples, output_dim)
        return np.random.randn(num_samples, 2)

# Create params dict with all necessary parameters for FunctionSample
params = {
    'name': 'TestFunctionSample',
    'verbose': True,
    'dumper': None
}

# Create a model instance
model = MockModel()

# Construct ARGS dict
ARGS = {
    'params': params,
    'verbose': True
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type FunctionSample — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a FunctionSample output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Timer Ran
ran on a synthesized input · constructed an object; no result was computed
rehoss/lbmpc_semimarkov/lbmpc_semimarkov/acq/acquisition.py
code served (permissive licence) · get_code("0ab042ab92eeb876")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Timer: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import logging
import time
import datetime
import tempfile
import os

# Set up logging to avoid issues
logging.basicConfig(level=logging.INFO)

# Create a temporary file for logging output
temp_dir = tempfile.gettempdir()
temp_logfile = os.path.join(temp_dir, "timer_test.log")

ARGS = {
    "name": "test_timer",
    "filename": temp_logfile,
    "level": logging.INFO,
    "verbose": True,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
none outside the standard library — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Timer — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Timer output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
AlgoAcqFunction Not yet run
unverified (harvested, no recorded run)
rehoss/lbmpc_semimarkov/lbmpc_semimarkov/acq/acquisition.py
code served (permissive licence) · get_code("9c847fe3e42ed3b1")
How it ran

no invocation recorded.

BaxAcqFunction Not yet run
unverified (harvested, no recorded run)
rehoss/lbmpc_semimarkov/lbmpc_semimarkov/acq/acquisition.py
code served (permissive licence) · get_code("3da2fb95b2eeaacf")
How it ran

no invocation recorded.

Also present in other papers' code

Of 8 samples lifted from this paper's linked repositories, 8 carry a function identity and 0 do not. Of the 8 distinct functions with an identity, 3 are also present in other papers whose code we hold (3 with at least one paper dated earlier, 0 the same month, 0 later) and 5 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
AcqFunction
brick of this paper's assembly: helper
1 other paper
1 dated earlier
2112.05244 (2021-12)
An Experimental Design Perspective on Model-Based Reinforcement Learning
Base
brick of this paper's assembly: helper
1 other paper
1 dated earlier · 1 only as an assembly brick
2112.05244 (2021-12)
An Experimental Design Perspective on Model-Based Reinforcement Learning
FunctionSample
brick of this paper's assembly: helper
1 other paper
1 dated earlier
2104.09460 (2021-04)
Bayesian Algorithm Execution: Estimating Computable Properties of Black-box Functions Using Mutual Information

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Many potential applications of reinforcement learning (RL) are stymied by the large numbers of samples required to learn an effective policy. This is especially true when applying RL to real-world control tasks, e.g. in the sciences or robotics, where executing a policy in the environment is costly. In popular RL algorithms, agents typically explore either by adding stochasticity to a reward-maximizing policy or by attempting to gather maximal information about environment dynamics without taking the given task into account. In this work, we develop a method that allows us to plan for exploration while taking both the task and the current knowledge about the dynamics into account. The key insight to our approach is to plan an action sequence that maximizes the expected information gain about the optimal trajectory for the task at hand. We demonstrate that our method learns strong policies with 2x fewer samples than strong exploration baselines and 200x fewer samples than model free methods on a diverse set of low-to-medium dimensional control tasks in both the open-loop and closed-loop control settings. 1

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2210.04642")
get_code_for_paper("2210.04642")
have("2210.04642")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2210.04642.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2210.04642.svg)](https://syntology.ai/paper/2210.04642)

Connect an agent — have() is free.