SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 1707.06347 · arXiv.org · 2017

Proximal Policy Optimization Algorithms

Alec Radford, John Schulman, Prafulla Dhariwal, Filip Wolski, Oleg Klimov

arXiv · PDF · Open in the Atlas

Code that ran

99 of 176 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (53 of them constructed an object rather than computing a result; 7 honoured the contract we drafted, 2 violated it and 62 have no contract verdict; 28 failures of our instrument; 77 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/1707.06347.json · badge: /api/ran/1707.06347.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
hmhuy0/SIM-RL community
not in the paper's declared repositories
10 ran · 2 no recorded run
mit-realm/neuriss community
not in the paper's declared repositories
8 ran · 4 no recorded run
ku2482/gail-airl-ppo.pytorch community
not in the paper's declared repositories
6 ran · 2 no recorded run
evieq01/oodil community
not in the paper's declared repositories
5 ran · 2 no recorded run
GiadaSimionato/Reasoning_Agents_2020 community
Papers with Code archive
4 ran · 3 no recorded run
bonniesjli/PPO-Reacher_UnityML community
not in the paper's declared repositories
4 ran · 0 no recorded run
dmiu-shell/deeprl-shell community
not in the paper's declared repositories
4 ran · 0 no recorded run
copy not recorded community
not in the paper's declared repositories
4 ran · 0 no recorded run
hamishs/JAX-RL community
not in the paper's declared repositories
3 ran · 2 no recorded run
Gregory-Eales/proximal-policy-optimization community
Papers with Code archive
3 ran · 1 no recorded run
Narsil/rl-baselines community
not in the paper's declared repositories
3 ran · 1 no recorded run
bonniesjli/PPO_Reacher community
not in the paper's declared repositories
3 ran · 0 no recorded run
nikhilbarhate99/PPO-PyTorch community
not in the paper's declared repositories
3 ran · 0 no recorded run
ASzot/ppo-pytorch community
Papers with Code archive
3 ran · 0 no recorded run
near32/regym community
not in the paper's declared repositories
3 ran · 0 no recorded run
bay3s/ppo-rl community
not in the paper's declared repositories
2 ran · 1 no recorded run
xtma/simple-pytorch-rl community
not in the paper's declared repositories
2 ran · 1 no recorded run
JonasRSV/PPO community
Papers with Code archive
2 ran · 1 no recorded run
silvialuu/DRL-2018 community
not in the paper's declared repositories
2 ran · 1 no recorded run
tilkb/thermoai community
not in the paper's declared repositories
2 ran · 0 no recorded run
mark-gluzman/NmodelPPO community
not in the paper's declared repositories
2 ran · 0 no recorded run
lcswillems/torch-ac community
not in the paper's declared repositories
1 ran · 3 no recorded run
wangzhengfei0730/NIPS2018-AIforProsthetics community
not in the paper's declared repositories
1 ran · 3 no recorded run
InSpaceAI/RL-Zoo community
Papers with Code archive
1 ran · 3 no recorded run
liuyuezhang/pyrl community
not in the paper's declared repositories
1 ran · 2 no recorded run
FunctionStatusWhere it lives
Actor Ran
ran on a synthesized input · constructed an object; no result was computed
bonniesjli/PPO-Reacher_UnityML/agent.py
pointer only (licence: NONE) · get_code("e9f1207fc08f9fa9")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, bonniesjli/PPO-Reacher_UnityML when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Actor Ran
ran on a synthesized input · constructed an object; no result was computed
bonniesjli/PPO_Reacher/agent.py
pointer only (licence: NONE) · get_code("8fc0ce07edd32744")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, bonniesjli/PPO_Reacher when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Actor Ran
ran on a synthesized input · constructed an object; no result was computed
Gregory-Eales/proximal-policy-optimization/ppo/modules/ppo.py
code served (permissive licence) · get_code("c3cb4b39b7e82812")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Actor: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Define the Actor arguments
actor_lr = 0.001
epsilon = 0.2

ARGS = {
    'actor_lr': actor_lr,
    'epsilon': epsilon
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Actor — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Actor output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ActorCritic Ran
ran on a synthesized input · constructed an object; no result was computed
bonniesjli/PPO-Reacher_UnityML/agent.py
pointer only (licence: NONE) · get_code("f7a59411ac390f78")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, bonniesjli/PPO-Reacher_UnityML when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

ActorCritic Ran
ran on a synthesized input · constructed an object; no result was computed
bonniesjli/PPO_Reacher/agent.py
pointer only (licence: NONE) · get_code("607bb439a428fd2a")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, bonniesjli/PPO_Reacher when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

ActorCritic Ran
ran on a synthesized input · constructed an object; no result was computed
nikhilbarhate99/PPO-PyTorch/PPO.py
code served (permissive licence) · get_code("bab8cee03856463f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ActorCritic: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    'state_dim': 4,
    'action_dim': 3,
    'has_continuous_action_space': True,
    'action_std_init': 0.6,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ActorCritic — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ActorCritic output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ActorCritic Ran
ran on a synthesized input
bay3s/ppo-rl/src/ppo/ppo.py
pointer only (licence: NONE) · get_code("6ff581679958bf77")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, bay3s/ppo-rl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

ActorCritic Ran
ran on a synthesized input
liuyuezhang/pyrl/ppo/agent.py
code served (permissive licence) · get_code("2f383d9f3f2f7b4f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
import torch.nn.functional as F

class ActorCritic(nn.Module):

    def __init__(self, state_dim, action_dim, n_latent_var=64):
        super(ActorCritic, self).__init__()
        self.linear1 = nn.Linear(state_dim, n_latent_var)
        self.linear2 = nn.Linear(n_latent_var, n_latent_var)
        self.actor = nn.Linear(n_latent_var, action_dim)
        self.critic = nn.Linear(n_latent_var, 1)

    def forward(self, x):
        x = F.relu(self.linear1(x))
        x = F.relu(self.linear2(x))
        return (F.softmax(self.actor(x), dim=-1), self.critic(x))


INIT_ARGS = {
    'state_dim': 4,
    'action_dim': 3,
    'n_latent_var': 64
}

ARGS = {
    'x': torch.randn(2, 4)
}

model = ActorCritic(**INIT_ARGS)
output = model(**ARGS)
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
ActorNet Ran
ran on a synthesized input
xtma/simple-pytorch-rl/ppo.py
pointer only (licence: NONE) · get_code("13b852ad71d18604")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, xtma/simple-pytorch-rl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Agent Ran
ran on a synthesized input · constructed an object; no result was computed
bonniesjli/PPO-Reacher_UnityML/agent.py
pointer only (licence: NONE) · get_code("b1f30e180cb81a3a")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, bonniesjli/PPO-Reacher_UnityML when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Agent Ran
ran on a synthesized input · constructed an object; no result was computed
bonniesjli/PPO_Reacher/agent.py
pointer only (licence: NONE) · get_code("4c40cb5849fd342f")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, bonniesjli/PPO_Reacher when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

BaseAgent Ran
ran on a synthesized input · constructed an object; no result was computed
dmiu-shell/deeprl-shell/deep_rl/agent/PPO_agent.py
code served (permissive licence) · get_code("38803d98e366da6d")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, BaseAgent: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
import numpy as np

torch.manual_seed(0)

class DummyNetwork(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(4, 3)
    
    def predict(self, state, to_numpy=False):
        output = self.linear(torch.tensor(state, dtype=torch.float32))
        if to_numpy:
            return output.detach().numpy()
        return output

class DummyEnv:
    def __init__(self):
        self.state_shape = (4,)
        self.action_space_n = 3
    
    def reset(self):
        return np.random.randn(4).astype(np.float32)
    
    def step(self, action):
        state = np.random.randn(4).astype(np.float32)
        reward = 1.0
        done = False
        info = {}
        return state, reward, done, info
    
    def close(self):
        pass

class DummyStateNormalizer:
    def __init__(self):
        self.read_only = False
    
    def __call__(self, state):
        return state.astype(np.float32)
    
    def set_read_only(self):
        self.read_only = True
    
    def unset_read_only(self):
        self.read_only = False

class DummyLogger:
    def info(self, msg):
        pass

class DummyConfig:
    def __init__(self):
        self.evaluation_env = DummyEnv()
        self.state_normalizer = DummyStateNormalizer()
        self.evaluation_episodes_interval = 0
        self.evaluation_episodes = 2
        self.logger = DummyLogger()

config = DummyConfig()
network = DummyNetwork()
config.network = network

ARGS = {
    'config': config
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type BaseAgent — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a BaseAgent output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Batcher Ran
ran on a synthesized input · constructed an object; no result was computed
dmiu-shell/deeprl-shell/deep_rl/agent/PPO_agent.py
code served (permissive licence) · get_code("6dbcf603fe9c4a6f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Batcher: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# Create small numpy arrays as data
# The Batcher expects data to be a list of arrays where each array has the same first dimension (num_entries)
data_array_1 = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]])  # shape (4, 3)
data_array_2 = np.array([[0.1, 0.2, 0.3], [0.4, 0.5, 0.6], [0.7, 0.8, 0.9], [1.0, 1.1, 1.2]])  # shape (4, 3)

batch_size = 2
data = [data_array_1, data_array_2]

ARGS = {
    "batch_size": batch_size,
    "data": data,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Batcher — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Batcher output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Buffer Ran
ran on a synthesized input · constructed an object; no result was computed
Gregory-Eales/proximal-policy-optimization/ppo/modules/ppo.py
code served (permissive licence) · get_code("5eac27c1ede5a3f1")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Buffer: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# The Buffer class doesn't take any constructor arguments
# It only has methods that are called on instances

ARGS = {}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Buffer — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Buffer output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Buffer Ran
ran on a synthesized input
bay3s/ppo-rl/src/ppo/ppo.py
pointer only (licence: NONE) · get_code("d0735a6f70135e3f")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, bay3s/ppo-rl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Categorical Ran
ran on a synthesized input · constructed an object; no result was computed
JonasRSV/PPO/ppo.py
pointer only (licence: NONE) · get_code("cc0a6d60ba0d615d")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, JonasRSV/PPO when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Continous Ran
ran on a synthesized input · constructed an object; no result was computed
JonasRSV/PPO/ppo.py
pointer only (licence: NONE) · get_code("1e06bd08fb825800")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, JonasRSV/PPO when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Critic Ran
ran on a synthesized input · constructed an object; no result was computed
bonniesjli/PPO-Reacher_UnityML/agent.py
pointer only (licence: NONE) · get_code("b4455ab1f545ae9c")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, bonniesjli/PPO-Reacher_UnityML when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Critic Ran
ran on a synthesized input · constructed an object; no result was computed
Gregory-Eales/proximal-policy-optimization/ppo/modules/ppo.py
code served (permissive licence) · get_code("f1685211d6effd06")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Critic: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# Create dummy states and rewards for the optimize method
# States should be images that can go through conv layers
# Conv1 expects input of shape (batch, 3, H, W)
# After all conv layers and reshape, we get (batch, 1024)
batch_size = 2
states = torch.randn(batch_size, 3, 64, 64)  # (batch, channels, height, width)
rewards = torch.randn(batch_size, 1)  # (batch, 1) for MSELoss

ARGS = {
    'critic_lr': 0.001,
    'critic_epochs': 1,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch, tqdm — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Critic — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Critic output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
CriticNet Ran
ran on a synthesized input · constructed an object; no result was computed
xtma/simple-pytorch-rl/ppo.py
pointer only (licence: NONE) · get_code("1a18a57c9dbc3655")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, xtma/simple-pytorch-rl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

DictList Ran
ran on a synthesized input · constructed an object; no result was computed
GiadaSimionato/Reasoning_Agents_2020/babyai_rb/babyai/rl/algos/ppo.py
pointer only (licence: NONE) · get_code("28919ed273c0c995")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, GiadaSimionato/Reasoning_Agents_2020 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

ExtraInfoCollector Ran
ran on a synthesized input · constructed an object; no result was computed
GiadaSimionato/Reasoning_Agents_2020/babyai_rb/babyai/rl/algos/ppo.py
pointer only (licence: NONE) · get_code("06c62bb9eacfe08c")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, GiadaSimionato/Reasoning_Agents_2020 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

MLP Ran
ran on a synthesized input · constructed an object; no result was computed
mit-realm/neuriss/neuriss/rl/ppo.py
code served (permissive licence) · get_code("f2a4b483cf41dcfd")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, MLP: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    'in_dim': 4,
    'out_dim': 3,
    'hidden_layers': (8, 6),
    'hidden_activation': nn.ReLU(),
    'output_activation': None,
    'init': True,
    'gain': 1.0,
    'limit_lip': False
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type MLP — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a MLP output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
MemoryPPO Ran
ran on a synthesized input · constructed an object; no result was computed
nric/ProximalPolicyOptimizationContinuousKeras/ppo_keras_continuous.py
pointer only (licence: NONE) · get_code("75bc51af9c0cf574")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, nric/ProximalPolicyOptimizationContinuousKeras when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Mlp Ran
ran on a synthesized input
ASzot/ppo-pytorch/model.py
code served (permissive licence) · get_code("088783306dda80fe")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

def init_layer(m):
    weight = m.weight.data
    weight.normal_(0, 1)
    weight *= 1.0 / torch.sqrt(weight.pow(2).sum(1, keepdim=True))
    nn.init.constant_(m.bias.data, 0)
    return m

class Mlp(nn.Module):

    def __init__(self, num_inputs):
        super().__init__()
        self.actor_hidden = nn.Sequential(init_layer(nn.Linear(num_inputs, 64)), nn.Tanh(), init_layer(nn.Linear(64, 64)), nn.Tanh())
        self.critic = nn.Sequential(init_layer(nn.Linear(num_inputs, 64)), nn.Tanh(), init_layer(nn.Linear(64, 64)), nn.Tanh(), init_layer(nn.Linear(64, 1)))
        self.train()

    def forward(self, inputs):
        return (self.actor_hidden(inputs), self.critic(inputs))


INIT_ARGS = {
    'num_inputs': 4
}

ARGS = {
    'inputs': torch.randn(2, 4)
}

if __name__ == '__main__':
    model = Mlp(**INIT_ARGS)
    output = model(**ARGS)
    print(f"Actor hidden output shape: {output[0].shape}")
    print(f"Critic output shape: {output[1].shape}")
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
ModelUpdate Ran
ran on a synthesized input · constructed an object; no result was computed
Narsil/rl-baselines/rl_baselines/ppo.py
pointer only (licence: NONE) · get_code("7f143a6ee276436b")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, Narsil/rl-baselines when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

MyPPO Ran
ran on a synthesized input
BrianPulfer/PapersReimplementations/src/rl/ppo/ppo.py
code served (permissive licence) · get_code("bb56e14071f60cba")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np
import torch
import torch.nn as nn
from torch.distributions.categorical import Categorical

class MyPPO(nn.Module):
    """Implementation of a PPO model. The same backbone is used to get actor and critic values."""

    def __init__(self, in_shape, n_actions, hidden_d=100, share_backbone=False):
        super(MyPPO, self).__init__()
        self.in_shape = in_shape
        self.n_actions = n_actions
        self.hidden_d = hidden_d
        self.share_backbone = share_backbone
        in_dim = np.prod(in_shape)

        def to_features():
            return nn.Sequential(nn.Flatten(), nn.Linear(in_dim, hidden_d), nn.ReLU(), nn.Linear(hidden_d, hidden_d), nn.ReLU())
        self.backbone = to_features() if self.share_backbone else nn.Identity()
        self.actor = nn.Sequential(nn.Identity() if self.share_backbone else to_features(), nn.Linear(hidden_d, hidden_d), nn.ReLU(), nn.Linear(hidden_d, n_actions), nn.Softmax(dim=-1))
        self.critic = nn.Sequential(nn.Identity() if self.share_backbone else to_features(), nn.Linear(hidden_d, hidden_d), nn.ReLU(), nn.Linear(hidden_d, 1))

    def forward(self, x):
        features = self.backbone(x)
        action = self.actor(features)
        value = self.critic(features)
        return (Categorical(action).sample(), action, value)


INIT_ARGS = {
    'in_shape': (4, 3),
    'n_actions': 5,
    'hidden_d': 16,
    'share_backbone': False
}

ARGS = {
    'x': torch.randn(2, 4, 3)
}

if __name__ == '__main__':
    model = MyPPO(**INIT_ARGS)
    output = model(**ARGS)
    print("Output types:", type(output[0]), type(output[1]), type(output[2]))
    print("Sample shape:", output[0].shape)
    print("Action probs shape:", output[1].shape)
    print("Value shape:", output[2].shape)
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
NormalizedMLP Ran
ran on a synthesized input · constructed an object; no result was computed
mit-realm/neuriss/neuriss/rl/ppo.py
code served (permissive licence) · get_code("1271c934496f2b53")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, NormalizedMLP: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Define parameters for NormalizedMLP
in_dim = 4
out_dim = 3
hidden_layers = (8, 6)

# Create input_mean and input_std tensors
input_mean = torch.zeros(in_dim)
input_std = torch.ones(in_dim)

# Create activation functions
hidden_activation = nn.ReLU()
output_activation = nn.Tanh()

# Boolean and float parameters
init = True
gain = 1.0

ARGS = {
    "in_dim": in_dim,
    "out_dim": out_dim,
    "input_mean": input_mean,
    "input_std": input_std,
    "hidden_layers": hidden_layers,
    "hidden_activation": hidden_activation,
    "output_activation": output_activation,
    "init": init,
    "gain": gain,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type NormalizedMLP — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a NormalizedMLP output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
PPO Ran
ran on a synthesized input · constructed an object; no result was computed
uvipen/super-mario-bros-ppo-pytorch/src/model.py
code served (permissive licence) · get_code("1ebfd098cf241f3c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, PPO: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# PPO.__init__ takes num_inputs and num_actions
# num_inputs: number of input channels for Conv2d
# num_actions: number of actions for the actor output

num_inputs = 4
num_actions = 6

ARGS = {
    "num_inputs": num_inputs,
    "num_actions": num_actions,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type PPO — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a PPO output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
PPO Ran
ran on a synthesized input · constructed an object; no result was computed
Narsil/rl-baselines/rl_baselines/ppo.py
pointer only (licence: NONE) · get_code("b48c75c1f791937c")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, Narsil/rl-baselines when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

PPO Ran
ran on a synthesized input · constructed an object; no result was computed
dyabel/handson_rl/ppo.py
pointer only (licence: NONE) · get_code("00bb1a30b070c324")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, dyabel/handson_rl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

PPO Ran
ran on a synthesized input · constructed an object; no result was computed
nikhilbarhate99/PPO-PyTorch/PPO.py
code served (permissive licence) · get_code("bd4134c19ff6f1eb")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, PPO: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from torch.distributions import MultivariateNormal, Categorical

torch.manual_seed(0)

device = torch.device('cpu')

ARGS = {
    'state_dim': 4,
    'action_dim': 3,
    'lr_actor': 0.0003,
    'lr_critic': 0.001,
    'gamma': 0.99,
    'K_epochs': 4,
    'eps_clip': 0.2,
    'has_continuous_action_space': True,
    'action_std_init': 0.6
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type PPO — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a PPO output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
PPO Ran
ran on a synthesized input · constructed an object; no result was computed
zjlab-ammi/llm4rl/algos/ppo.py
pointer only (licence: NONE) · get_code("20cf521175a91e97")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, zjlab-ammi/llm4rl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

PPO Ran
ran on a synthesized input · constructed an object; no result was computed
hamishs/JAX-RL/src/jax_rl/algorithms/ppo.py
code served (permissive licence) · get_code("6e26f9d72a957989")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, PPO: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import jax
import jax.numpy as jnp
import haiku as hk
import optax

# Create simple Haiku modules for policy and value networks
def policy_fn(s):
    """Simple policy network that outputs action probabilities."""
    net = hk.Sequential([
        hk.Linear(8),
        jax.nn.relu,
        hk.Linear(3)  # 3 actions
    ])
    logits = net(s)
    return jax.nn.softmax(logits)

def value_fn(s):
    """Simple value network that outputs state value."""
    net = hk.Sequential([
        hk.Linear(8),
        jax.nn.relu,
        hk.Linear(1)
    ])
    return net(s)

# Transform Haiku functions into modules
policy = hk.transform(policy_fn)
value = hk.transform(value_fn)

# Define arguments for PPO
ARGS = {
    'key': jax.random.PRNGKey(0),
    'n_states': 4,
    'n_actions': 3,
    'gamma': 0.99,
    'lmda': 0.95,
    'epsilon': 0.2,
    'v_weight': 0.5,
    'e_weight': 0.01,
    'buffer_size': 16,
    'policy': policy,
    'value': value,
    'lr_policy': 0.001,
    'lr_value': 0.001
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
haiku, jax, numpy, optax — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type PPO — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a PPO output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
PPO Ran
ran on a synthesized input · constructed an object; no result was computed
tilkb/thermoai/controller/RL/PPO.py
pointer only (licence: NONE) · get_code("e33afa03c0df2bff")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, tilkb/thermoai when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

PPOAgent Ran
ran on a synthesized input · constructed an object; no result was computed
dmiu-shell/deeprl-shell/deep_rl/agent/PPO_agent.py
code served (permissive licence) · get_code("e3bd893b00cce00e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, PPOAgent: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
import numpy as np

torch.manual_seed(0)
np.random.seed(0)

class SimpleLogger:
    def info(self, msg):
        print(msg)

class SimpleNormalizer:
    def __init__(self):
        self.read_only = False
    
    def __call__(self, x):
        return x.astype(np.float32) if isinstance(x, np.ndarray) else x
    
    def set_read_only(self):
        self.read_only = True
    
    def unset_read_only(self):
        self.read_only = False

class SimpleEnv:
    def __init__(self):
        self.state_dim = 4
        self.action_dim = 3
    
    def reset(self):
        return np.random.randn(2, 4).astype(np.float32)
    
    def step(self, actions):
        batch_size = 2
        next_states = np.random.randn(batch_size, 4).astype(np.float32)
        rewards = np.random.randn(batch_size).astype(np.float32)
        terminals = np.zeros(batch_size, dtype=bool)
        info = [None] * batch_size
        return next_states, rewards, terminals, info

class SimpleNetwork(nn.Module):
    def __init__(self, state_dim, action_dim):
        super().__init__()
        self.state_dim = state_dim
        self.action_dim = action_dim
        self.fc = nn.Linear(state_dim, 128)
        self.actor = nn.Linear(128, action_dim)
        self.critic = nn.Linear(128, 1)
    
    def predict(self, states, to_numpy=False):
        if isinstance(states, np.ndarray):
            states = torch.from_numpy(states).float()
        
        x = torch.relu(self.fc(states))
        logits = self.actor(x)
        values = self.critic(x).squeeze(-1)
        
        actions = torch.argmax(logits, dim=-1)
        log_probs = torch.log_softmax(logits, dim=-1)
        log_probs = torch.gather(log_probs, -1, actions.unsqueeze(-1)).squeeze(-1)
        
        if to_numpy:
            return logits.detach().numpy()
        
        return actions, log_probs, logits, values

def make_task():
    return SimpleEnv()

def make_network(state_dim, action_dim):
    return SimpleNetwork(state_dim, action_dim)

def make_optimizer(params):
    return torch.optim.Adam(params, lr=0.001)

class Config:
    DEVICE = 'cpu'
    
    def __init__(self):
        self.task_fn = make_task
        self.network_fn = make_network
        self.optimizer_fn = make_optimizer
        self.state_normalizer = SimpleNormalizer()
        self.reward_normalizer = SimpleNormalizer()
        self.logger = SimpleLogger()
        self.evaluation_env = SimpleEnv()
        self.num_workers = 2
        self.rollout_length = 3
        self.evaluation_episodes_interval = None
        self.evaluation_episodes = 2

config = Config()

ARGS = {
    'config': config
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type PPOAgent — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a PPOAgent output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
PPOAgent Ran
ran on a synthesized input · constructed an object; no result was computed
ifestus/rl/policy_agents.py
pointer only (licence: NONE) · get_code("786ef0f1e1c69fc2")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ifestus/rl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

PPOAlgorithm Ran
ran on a synthesized input · constructed an object; no result was computed
near32/regym/regym/rl_algorithms/algorithms/PPO/ppo.py
code served (permissive licence) · get_code("08c8acc6b144eabd")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, PPOAlgorithm: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np

torch.manual_seed(0)

# Create a minimal model that the PPOAlgorithm expects
class MinimalPolicyValueNet(nn.Module):
    def __init__(self, state_dim=4, action_dim=3):
        super().__init__()
        self.fc1 = nn.Linear(state_dim, 8)
        self.policy_head = nn.Linear(8, action_dim)
        self.value_head = nn.Linear(8, 1)
    
    def forward(self, x):
        x = torch.relu(self.fc1(x))
        policy_logits = self.policy_head(x)
        value = self.value_head(x)
        return policy_logits, value

class MinimalIntrinsicModel(nn.Module):
    def __init__(self, state_dim=4, hidden_dim=8):
        super().__init__()
        self.fc = nn.Linear(state_dim, hidden_dim)
        self.output = nn.Linear(hidden_dim, 8)
    
    def forward(self, x):
        return self.output(torch.relu(self.fc(x)))

# Create model instances
model = MinimalPolicyValueNet(state_dim=4, action_dim=3)
target_intr_model = MinimalIntrinsicModel(state_dim=4)
predict_intr_model = MinimalIntrinsicModel(state_dim=4)

# Create optimizer
optimizer = optim.Adam(model.parameters(), lr=0.001, eps=1e-5)

# Create kwargs dict with all necessary configuration
kwargs = {
    'nbr_actor': 4,
    'use_cuda': False,
    'horizon': 128,
    'discount': 0.99,
    'use_gae': True,
    'gae_tau': 0.95,
    'entropy_weight': 0.01,
    'gradient_clip': 0.5,
    'optimization_epochs': 3,
    'mini_batch_size': 32,
    'ppo_ratio_clip': 0.2,
    'learning_rate': 0.001,
    'adam_eps': 1e-5,
    'use_vae': False,
    'rnd_update_period_running_meanstd_obs': 100,
    'rnd_update_period_running_meanstd_int_reward': 100,
}

# Summary writer can be None
sum_writer = None

ARGS = {
    'kwargs': kwargs,
    'model': model,
    'optimizer': optimizer,
    'target_intr_model': target_intr_model,
    'predict_intr_model': predict_intr_model,
    'sum_writer': sum_writer,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
ipdb, numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type PPOAlgorithm — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a PPOAlgorithm output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
PPONetwork Ran
ran on a synthesized input
jsztompka/PPO-demo/model.py
pointer only (licence: NONE) · get_code("7dbfda361c907b1e")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, jsztompka/PPO-demo when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

ParallelEnv Ran
ran on a synthesized input · constructed an object; no result was computed
GiadaSimionato/Reasoning_Agents_2020/babyai_rb/babyai/rl/algos/ppo.py
pointer only (licence: NONE) · get_code("f0b71d0c5374efdf")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, GiadaSimionato/Reasoning_Agents_2020 when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

ParallelEnv Ran
ran on a synthesized input · constructed an object; no result was computed
lcswillems/torch-ac/torch_ac/algos/ppo.py
code served (permissive licence) · get_code("028cd0cc569e4945")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ParallelEnv: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import gymnasium as gym
import numpy as np

# Create small stand-in environments using gymnasium
class SimpleEnv(gym.Env):
    """Minimal environment for testing."""
    
    def __init__(self):
        self.observation_space = gym.spaces.Box(low=0, high=1, shape=(4,), dtype=np.float32)
        self.action_space = gym.spaces.Discrete(2)
        self.step_count = 0
    
    def reset(self, seed=None):
        super().reset(seed=seed)
        self.step_count = 0
        obs = self.observation_space.sample()
        return obs, {}
    
    def step(self, action):
        self.step_count += 1
        obs = self.observation_space.sample()
        reward = float(action)
        terminated = self.step_count >= 10
        truncated = False
        info = {}
        return obs, reward, terminated, truncated, info
    
    def render(self):
        pass
    
    def close(self):
        pass

# Create multiple environment instances
env1 = SimpleEnv()
env2 = SimpleEnv()
env3 = SimpleEnv()

envs = [env1, env2, env3]

ARGS = {
    'envs': envs
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
gymnasium, numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ParallelEnv — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ParallelEnv output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Policy Ran
ran on a synthesized input · constructed an object; no result was computed
tuanpnm99/RLPongAgent/Agent.py
pointer only (licence: NONE) · get_code("c02c0631708465dc")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, tuanpnm99/RLPongAgent when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Policy Ran
ran on a synthesized input · constructed an object; no result was computed
mark-gluzman/NmodelPPO/actor_utils.py
pointer only (licence: NONE) · get_code("ec594f35cdf2f00e")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, mark-gluzman/NmodelPPO when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Policy Ran
ran on a synthesized input
ASzot/ppo-pytorch/model.py
code served (permissive licence) · get_code("be3a23c2176d855c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

def init_layer(m):
    weight = m.weight.data
    weight.normal_(0, 1)
    weight *= 1.0 / torch.sqrt(weight.pow(2).sum(1, keepdim=True))
    nn.init.constant_(m.bias.data, 0)
    return m

class Mlp(nn.Module):

    def __init__(self, num_inputs):
        super().__init__()
        self.actor_hidden = nn.Sequential(init_layer(nn.Linear(num_inputs, 64)), nn.Tanh(), init_layer(nn.Linear(64, 64)), nn.Tanh())
        self.critic = nn.Sequential(init_layer(nn.Linear(num_inputs, 64)), nn.Tanh(), init_layer(nn.Linear(64, 64)), nn.Tanh(), init_layer(nn.Linear(64, 1)))
        self.train()

    def forward(self, inputs):
        return (self.actor_hidden(inputs), self.critic(inputs))

class Policy(nn.Module):

    def __init__(self, obs_shape, action_space):
        super().__init__()
        self.actor_critic = Mlp(obs_shape[0])
        num_outputs = action_space.shape[0]
        self.action_mean = init_layer(nn.Linear(64, num_outputs))
        self.action_log_std = nn.Parameter(torch.zeros(1, num_outputs))

    def __get_dist(self, actor_features):
        action_mean = self.action_mean(actor_features)
        action_log_std = self.action_log_std.expand_as(action_mean)
        return torch.distributions.Normal(action_mean, action_log_std.exp())

    def act(self, inputs, deterministic=False):
        actor_features, value = self.actor_critic(inputs)
        dist = self.__get_dist(actor_features)
        if deterministic:
            action = dist.mean
        else:
            action = dist.sample()
        action_log_probs = dist.log_prob(action).sum(-1, keepdim=True)
        dist_entropy = dist.entropy().sum(-1).mean()
        return (value, action, action_log_probs)

    def get_value(self, inputs):
        _, value = self.actor_critic(inputs)
        return value

    def evaluate_actions(self, inputs, action):
        actor_features, value = self.actor_critic(inputs)
        dist = self.__get_dist(actor_features)
        action_log_probs = dist.log_prob(action).sum(-1, keepdim=True)
        dist_entropy = dist.entropy().sum(-1).mean()
        return (value, action_log_probs, dist_entropy)

class ActionSpace:
    def __init__(self, shape):
        self.shape = shape

INIT_ARGS = {
    'obs_shape': (4,),
    'action_space': ActionSpace((3,))
}

ARGS = {
    'inputs': torch.randn(2, 4),
    'deterministic': False
}

METHOD = 'act'
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
PolicyNN Ran
ran on a synthesized input · constructed an object; no result was computed
mark-gluzman/NmodelPPO/actor_utils.py
pointer only (licence: NONE) · get_code("06f3c47cdf2dd979")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, mark-gluzman/NmodelPPO when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

PolicyUpdate Ran
ran on a synthesized input · constructed an object; no result was computed
Narsil/rl-baselines/rl_baselines/ppo.py
pointer only (licence: NONE) · get_code("2eb0c930a45a7472")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, Narsil/rl-baselines when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

ReplayMemory Ran
ran on a synthesized input · constructed an object; no result was computed
llSourcell/OpenAI_Five_vs_Dota2_Explained/ppo.py
code served (permissive licence) · get_code("7539f57b4803c98e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ReplayMemory: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import random
import torch

torch.manual_seed(0)

# ReplayMemory stores events and samples from them
# The __init__ takes capacity as a parameter
# push() expects events that can be zipped together
# sample() samples batch_size items and returns concatenated tensors

# Create sample events: tuples of (state, action, reward, next_state, done)
# Each component should be a tensor for proper concatenation in sample()
state_1 = torch.randn(1, 4)
action_1 = torch.tensor([[0]])
reward_1 = torch.tensor([[1.0]])
next_state_1 = torch.randn(1, 4)
done_1 = torch.tensor([[0]])

state_2 = torch.randn(1, 4)
action_2 = torch.tensor([[1]])
reward_2 = torch.tensor([[0.5]])
next_state_2 = torch.randn(1, 4)
done_2 = torch.tensor([[1]])

state_3 = torch.randn(1, 4)
action_3 = torch.tensor([[0]])
reward_3 = torch.tensor([[2.0]])
next_state_3 = torch.randn(1, 4)
done_3 = torch.tensor([[0]])

# events is expected to be a tuple of sequences that can be zipped
events = (
    [state_1, state_2, state_3],
    [action_1, action_2, action_3],
    [reward_1, reward_2, reward_3],
    [next_state_1, next_state_2, next_state_3],
    [done_1, done_2, done_3]
)

ARGS = {
    "capacity": 10
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ReplayMemory — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ReplayMemory output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
RolloutBuffer Ran
ran on a synthesized input · constructed an object; no result was computed
mit-realm/neuriss/neuriss/rl/ppo.py
code served (permissive licence) · get_code("049dd1dd7386f7f1")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, RolloutBuffer: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)
np.random.seed(0)

# Define parameters for RolloutBuffer
buffer_size = 8
state_dim = 4
action_dim = 3
device = torch.device('cpu')
mix = 2

ARGS = {
    'buffer_size': buffer_size,
    'state_dim': state_dim,
    'action_dim': action_dim,
    'device': device,
    'mix': mix,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type RolloutBuffer — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a RolloutBuffer output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
RolloutBuffer Ran
ran on a synthesized input · constructed an object; no result was computed
ku2482/gail-airl-ppo.pytorch/gail_airl_ppo/algo/ppo.py
code served (permissive licence) · get_code("168b3d776129e246")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, RolloutBuffer: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)
np.random.seed(0)

ARGS = {
    'buffer_size': 8,
    'state_shape': (4,),
    'action_shape': (3,),
    'device': 'cpu',
    'mix': 2,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type RolloutBuffer — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a RolloutBuffer output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
RolloutBuffer Ran
ran on a synthesized input · constructed an object; no result was computed
nikhilbarhate99/PPO-PyTorch/PPO.py
code served (permissive licence) · get_code("22bcf7bc55b864b8")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, RolloutBuffer: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

# RolloutBuffer.__init__ takes no parameters beyond self
# The function body initializes empty lists for various attributes
# No parameters are required for instantiation

ARGS = {}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type RolloutBuffer — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a RolloutBuffer output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
RolloutBuffer Ran
ran on a synthesized input · constructed an object; no result was computed
evieq01/oodil/simulated_robot/gail_airl_ppo/algo/ppo.py
code served (permissive licence) · get_code("c169652ebc3cb1c4")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, RolloutBuffer: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)
np.random.seed(0)

ARGS = {
    "buffer_size": 8,
    "state_shape": (4,),
    "action_shape": (3,),
    "device": torch.device("cpu"),
    "mix": 2,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type RolloutBuffer — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a RolloutBuffer output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
RolloutBuffer_PPO_lag Ran
ran on a synthesized input · constructed an object; no result was computed
hmhuy0/SIM-RL/Sources/algo/ppo.py
pointer only (licence: NONE) · get_code("6f986aa2d59a44f8")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, hmhuy0/SIM-RL when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Runner Ran
ran on a synthesized input · constructed an object; no result was computed
wangzhengfei0730/NIPS2018-AIforProsthetics/baselines/ppo2/ppo2.py
pointer only (licence: NONE) · get_code("c80e9a362666d32e")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, wangzhengfei0730/NIPS2018-AIforProsthetics when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

StateFunction Ran
ran on a synthesized input · constructed an object; no result was computed
hmhuy0/SIM-RL/Sources/algo/ppo.py
pointer only (licence: NONE) · get_code("a78ee4761e8df9d8")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, hmhuy0/SIM-RL when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

StateFunction Ran
ran on a synthesized input · constructed an object; no result was computed
ku2482/gail-airl-ppo.pytorch/gail_airl_ppo/algo/ppo.py
code served (permissive licence) · get_code("7f9dae876ba7e00b")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, StateFunction: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    "state_shape": (4,),
    "hidden_units": (64, 64),
    "hidden_activation": nn.Tanh()
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type StateFunction — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a StateFunction output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
StateFunction Ran
ran on a synthesized input · constructed an object; no result was computed
evieq01/oodil/simulated_robot/gail_airl_ppo/algo/ppo.py
code served (permissive licence) · get_code("ab47bdba4f875268")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, StateFunction: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    "state_shape": (4,),
    "hidden_units": (64, 64),
    "hidden_activation": nn.Tanh()
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type StateFunction — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a StateFunction output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
StateIndependentPolicy Ran
ran on a synthesized input · constructed an object; no result was computed
hmhuy0/SIM-RL/Sources/algo/ppo.py
pointer only (licence: NONE) · get_code("fbd28096c7f0c30f")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, hmhuy0/SIM-RL when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

StateIndependentPolicy Ran
ran on a synthesized input · constructed an object; no result was computed
ku2482/gail-airl-ppo.pytorch/gail_airl_ppo/algo/ppo.py
code served (permissive licence) · get_code("dea2715df3f6a1e3")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, StateIndependentPolicy: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn
import math

torch.manual_seed(0)

# Define state_shape and action_shape as tuples
# state_shape[0] is used as input_dim, action_shape[0] is used as output_dim
state_shape = (4,)  # state dimension
action_shape = (3,)  # action dimension

# hidden_units parameter
hidden_units = (64, 64)

# hidden_activation parameter
hidden_activation = nn.Tanh()

ARGS = {
    "state_shape": state_shape,
    "action_shape": action_shape,
    "hidden_units": hidden_units,
    "hidden_activation": hidden_activation,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type StateIndependentPolicy — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a StateIndependentPolicy output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
StateIndependentPolicy Ran
ran on a synthesized input · constructed an object; no result was computed
evieq01/oodil/simulated_robot/gail_airl_ppo/algo/ppo.py
code served (permissive licence) · get_code("181b4202198be200")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, StateIndependentPolicy: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn
import math

torch.manual_seed(0)

ARGS = {
    "state_shape": (4,),
    "action_shape": (3,),
    "hidden_units": (64, 64),
    "hidden_activation": nn.Tanh(),
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type StateIndependentPolicy — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a StateIndependentPolicy output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Storage Ran
ran on a synthesized input · constructed an object; no result was computed
near32/regym/regym/rl_algorithms/algorithms/PPO/ppo.py
code served (permissive licence) · get_code("e626bd567c81f207")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Storage: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)

ARGS = {
    'keys': None
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Storage — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Storage output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block

Showing the first 60. An agent gets every one from get_harvested_code_for_paper("1707.06347").

Also present in other papers' code

Of 176 samples lifted from this paper's linked repositories, 176 carry a function identity and 0 do not. Of the 170 distinct functions with an identity, 26 are also present in other papers whose code we hold (9 with at least one paper dated earlier, 0 the same month, 17 later) and 144 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
constfn
brick of this paper's assembly: helper
8 other papers
8 later
1806.07857 (2018-06)
RUDDER: Return Decomposition for Delayed Rewards
1812.02341 (2018-12)
Quantifying Generalization in Reinforcement Learning
1902.00255 (2019-02)
Policy Consolidation for Continual Reinforcement Learning
and 5 more not named here
sf01
brick of this paper's assembly: data_transform
8 other papers
8 later
1806.07857 (2018-06)
RUDDER: Return Decomposition for Delayed Rewards
1812.02341 (2018-12)
Quantifying Generalization in Reinforcement Learning
1902.00255 (2019-02)
Policy Consolidation for Continual Reinforcement Learning
and 5 more not named here
safemean
brick of this paper's assembly: metric
6 other papers
6 later · 1 only as an assembly brick
1804.06459 (2018-04)
On Learning Intrinsic Rewards for Policy Gradient Methods
1905.02363 (2019-05)
Dimension-Wise Importance Sampling Weight Clipping for Sample-Efficient Reinforcement Learning
1910.09191 (2019-10)
Regularization Matters in Policy Optimization
and 3 more not named here
build_mlp
brick of this paper's assembly: helper
3 other papers
3 later · 1 only as an assembly brick
1801.01290 (2018-01)
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
1907.00953 (2019-07)
Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model
2109.05940 (2021-09)
Cross Domain Robot Imitation with Invariant Representation
BaseAgent
brick of this paper's assembly: helper
2 other papers
2 dated earlier · 2 only as an assembly brick
1509.02971 (2015-09)
Continuous control with deep reinforcement learning
1509.06461 (2015-09)
Deep Reinforcement Learning with Double Q-learning
Buffer
brick of this paper's assembly: helper
2 other papers
2 dated earlier · 2 only as an assembly brick
1509.02971 (2015-09)
Continuous control with deep reinforcement learning
1509.06461 (2015-09)
Deep Reinforcement Learning with Double Q-learning
calculate_gae
brick of this paper's assembly: helper
2 other papers
2 later · 1 only as an assembly brick
2109.05940 (2021-09)
Cross Domain Robot Imitation with Invariant Representation
2406.19626 (2024-06)
Safety through feedback in Constrained RL
Agent
brick of this paper's assembly: helper
1 other paper
1 later
1810.12894 (2018-10)
Exploration by Random Network Distillation
Algorithm
brick of this paper's assembly: helper
1 other paper
1 later
1801.01290 (2018-01)
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
BaseAlgo
brick of this paper's assembly: helper
1 other paper
1 dated earlier · 1 only as an assembly brick
1602.01783 (2016-02)
Asynchronous Methods for Deep Reinforcement Learning
Critic
brick of this paper's assembly: model
1 other paper
1 dated earlier · 1 only as an assembly brick
1602.01783 (2016-02)
Asynchronous Methods for Deep Reinforcement Learning
DictList
brick of this paper's assembly: helper
1 other paper
1 dated earlier · 1 only as an assembly brick
1602.01783 (2016-02)
Asynchronous Methods for Deep Reinforcement Learning
ExtraInfoCollector
brick of this paper's assembly: helper
1 other paper
1 later
2302.02662 (2023-02)
Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning
PPOModel
brick of this paper's assembly: model
1 other paper
1 later
1810.12894 (2018-10)
Exploration by Random Network Distillation
ParallelEnv
brick of this paper's assembly: helper
1 other paper
1 dated earlier · 1 only as an assembly brick
1602.01783 (2016-02)
Asynchronous Methods for Deep Reinforcement Learning
StateIndependentPolicy
brick of this paper's assembly: helper
1 other paper
1 later
2406.19626 (2024-06)
Safety through feedback in Constrained RL
TrajectoryBuffer
brick of this paper's assembly: helper
1 other paper
1 dated earlier · 1 only as an assembly brick
1509.02971 (2015-09)
Continuous control with deep reinforcement learning
atanh
brick of this paper's assembly: helper
1 other paper
1 later · 1 only as an assembly brick
1801.01290 (2018-01)
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
calculate_gae_cost
brick of this paper's assembly: helper
1 other paper
1 later · 1 only as an assembly brick
2406.19626 (2024-06)
Safety through feedback in Constrained RL
calculate_log_pi
brick of this paper's assembly: helper
1 other paper
1 later · 1 only as an assembly brick
1801.01290 (2018-01)
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

Shown: the 20 functions present in the most other papers. 6 more shared functions are not listed here.

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

More are linked than shown here.

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

We propose a new family of policy gradient methods for reinforcement learning, which alternate between sampling data through interaction with the environment, and optimizing a "surrogate" objective function using stochastic gradient ascent. Whereas standard policy gradient methods perform one gradient update per data sample, we propose a novel objective function that enables multiple epochs of minibatch updates. The new methods, which we call proximal policy optimization (PPO), have some of the benefits of trust region policy optimization (TRPO), but they are much simpler to implement, more general, and have better sample complexity (empirically). Our experiments test PPO on a collection of benchmark tasks, including simulated robotic locomotion and Atari game playing, and we show that PPO outperforms other online policy gradient methods, and overall strikes a favorable balance between sample complexity, simplicity, and wall-time.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("1707.06347")
get_code_for_paper("1707.06347")
have("1707.06347")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/1707.06347.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/1707.06347.svg)](https://syntology.ai/paper/1707.06347)

Connect an agent — have() is free.