SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 1801.01290 · ICML · 2018

Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

Sergey Levine, Pieter Abbeel, Aurick Zhou, Tuomas Haarnoja

arXiv · PDF · Open in the Atlas

Code that ran

91 of 148 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (61 of them constructed an object rather than computing a result; 3 honoured the contract we drafted, 1 violated it and 69 have no contract verdict; 18 failures of our instrument; 57 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/1801.01290.json · badge: /api/ran/1801.01290.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
yhisaki/average-reward-drl community
not in the paper's declared repositories
10 ran · 3 no recorded run
BY571/Soft-Actor-Critic-and-Extensions community
not in the paper's declared repositories
9 ran · 1 no recorded run
kdally/fault-tolerant-flight-control-drl community
not in the paper's declared repositories
8 ran · 2 no recorded run
thomashirtz/soft-actor-critic community
not in the paper's declared repositories
7 ran · 5 no recorded run
MatthieuSarkis/Portfolio-Optimization-and-Goal-Based-Investment-with-Reinforcement-Learning community
Papers with Code archive
6 ran · 1 no recorded run
SaminYeasar/off_policy_ac community
Papers with Code archive
5 ran · 1 no recorded run
ku2482/discor.pytorch community
not in the paper's declared repositories
4 ran · 7 no recorded run
garyzyr001/rethinking-airl community
not in the paper's declared repositories
4 ran · 4 no recorded run
tmjeong1103/RL_with_RAY community
not in the paper's declared repositories
4 ran · 0 no recorded run
MarsEleven/car_racer_RL community
Papers with Code archive
3 ran · 4 no recorded run
Rafael1s/Deep-Reinforcement-Learning-Udacity community
Papers with Code archive
3 ran · 3 no recorded run
h-aboutalebi/SparceReward community
not in the paper's declared repositories
3 ran · 1 no recorded run
sunfex/weighted-sac community
not in the paper's declared repositories
3 ran · 1 no recorded run
baturaysaglam/la3p community
Papers with Code archive
3 ran · 1 no recorded run
roythuly/obac community
not in the paper's declared repositories
3 ran · 1 no recorded run
quantumiracle/Popular-RL-Algorithms community
not in the paper's declared repositories
3 ran · 1 no recorded run
ajaysub110/rl-pytorch community
Papers with Code archive
2 ran · 5 no recorded run
watchernyu/spinningup-drl-prototyping community
not in the paper's declared repositories
2 ran · 4 no recorded run
ku2482/gail-airl-ppo.pytorch community
not in the paper's declared repositories
2 ran · 3 no recorded run
Kaixhin/imitation-learning community
Papers with Code archive
2 ran · 0 no recorded run
ShawK91/Evolutionary-Reinforcement-Learning community
Papers with Code archive
1 ran · 2 no recorded run
kushagra06/SAC community
not in the paper's declared repositories
1 ran · 1 no recorded run
hyunin-lee/ForecasterSAC community
not in the paper's declared repositories
1 ran · 1 no recorded run
autumnwu/aggressive-q-learning-with-ensembles community
Papers with Code archive
1 ran · 0 no recorded run
tarod13/SAC community
not in the paper's declared repositories
1 ran · 0 no recorded run
FunctionStatusWhere it lives
AbstractAgent Ran
ran on a synthesized input · constructed an object; no result was computed
h-aboutalebi/SparceReward/engine/algorithms/SAC/sac.py
pointer only (licence: NONE) · get_code("424bf06f8d8eb998")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, h-aboutalebi/SparceReward when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Actor Ran
ran on a synthesized input · constructed an object; no result was computed
MatthieuSarkis/Portfolio-Optimization-and-Goal-Based-Investment-with-Reinforcement-Learning/src/agents.py
code served (permissive licence) · get_code("de5e64dc2bfc6a33")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Actor: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
import tempfile
import os

torch.manual_seed(0)

# Create a temporary directory for checkpoints
temp_dir = tempfile.mkdtemp()

ARGS = {
    'lr_pi': 0.001,
    'max_actions': np.array([1.0, 1.0, 1.0]),
    'action_space_dimension': 3,
    'log_sigma_min': -20.0,
    'log_sigma_max': 2.0,
    'input_shape': (4,),
    'layer_neurons': 8,
    'network_name': 'test_actor',
    'checkpoint_directory_networks': temp_dir,
    'device': 'cpu'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Actor — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Actor output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Actor Ran
ran on a synthesized input · constructed an object; no result was computed
BY571/Soft-Actor-Critic-and-Extensions/files/Agent.py
code served (permissive licence) · get_code("5d8f81c210a0aacb")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Actor: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    "state_size": 4,
    "action_size": 3,
    "seed": 42,
    "device": torch.device("cpu"),
    "hidden_size": 32,
    "init_w": 0.003,
    "log_std_min": -20,
    "log_std_max": 2,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Actor — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Actor output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Actor Ran
ran on a synthesized input · constructed an object; no result was computed
SaminYeasar/off_policy_ac/SAC/SAC.py
pointer only (licence: NONE) · get_code("dc01224a73ea8148")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, SaminYeasar/off_policy_ac when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Agent_ManualTemperature Ran
ran on a synthesized input · constructed an object; no result was computed
MatthieuSarkis/Portfolio-Optimization-and-Goal-Based-Investment-with-Reinforcement-Learning/src/agents.py
code served (permissive licence) · get_code("0918147b61a18074")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Agent_ManualTemperature: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import gym
import numpy as np
from typing import Tuple

torch.manual_seed(0)
np.random.seed(0)

# Create a minimal gym environment
env = gym.make('Pendulum-v1')

# Create minimal stub classes for dependencies
class ReplayBuffer:
    def __init__(self, size, input_shape, action_space_dimension):
        self.size = size
        self.input_shape = input_shape
        self.action_space_dimension = action_space_dimension
    
    def push(self, state, action, reward, new_state, done):
        pass

class Actor(torch.nn.Module):
    def __init__(self, lr_pi, action_space_dimension, max_actions, input_shape, 
                 layer_neurons, network_name, checkpoint_directory_networks, device):
        super().__init__()
        self.lr_pi = lr_pi
        self.action_space_dimension = action_space_dimension
        self.max_actions = max_actions
        self.input_shape = input_shape
        self.layer_neurons = layer_neurons
        self.network_name = network_name
        self.checkpoint_directory_networks = checkpoint_directory_networks
        self.device = device
        
        # Minimal network architecture
        input_dim = input_shape[0] if isinstance(input_shape, tuple) else input_shape
        self.fc1 = torch.nn.Linear(input_dim, layer_neurons)
        self.fc2 = torch.nn.Linear(layer_neurons, action_space_dimension)
    
    def sample(self, state, reparameterize=False):
        x = torch.relu(self.fc1(state))
        actions = torch.tanh(self.fc2(x))
        log_probs = torch.zeros(state.shape[0])
        return actions, log_probs
    
    def forward(self, state):
        x = torch.relu(self.fc1(state))
        return torch.tanh(self.fc2(x))

# Inject the stub classes into the module namespace
import sys
current_module = sys.modules[__name__]
setattr(current_module, 'ReplayBuffer', ReplayBuffer)
setattr(current_module, 'Actor', Actor)

ARGS = {
    'lr_Q': 0.001,
    'lr_pi': 0.001,
    'input_shape': (4,),
    'tau': 0.005,
    'env': env,
    'checkpoint_directory_networks': '/tmp/checkpoints',
    'gamma': 0.99,
    'size': 1000000,
    'layer_size': 256,
    'batch_size': 256,
    'delay': 1,
    'grad_clip': 1.0,
    'device': 'cpu'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
gym, numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Agent_ManualTemperature — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Agent_ManualTemperature output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
BaseQNet Ran
ran on a synthesized input · constructed an object; no result was computed
sunfex/weighted-sac/sac.py
code served (permissive licence) · get_code("d6b54769dcaa3ec5")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, BaseQNet: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    'state_dim': 4,
    'act_dim': 3,
    'hidden_dim': 8,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type BaseQNet — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a BaseQNet output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Batch Ran
ran on a synthesized input · constructed an object; no result was computed
yhisaki/average-reward-drl/average_reward_drl/algorithms/sac.py
pointer only (licence: NONE) · get_code("6bb425a4771dc88f")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, yhisaki/average-reward-drl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

BatchQNet Ran
ran on a synthesized input · constructed an object; no result was computed
sunfex/weighted-sac/sac.py
code served (permissive licence) · get_code("e6f3bef6186ec524")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, BatchQNet: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

# Define small dimensions for testing
obs_dim = 4
act_dim = 3
hidden_dim = 8
N = 2

# Create sample tensors with batch size 2
batch_size = 2
states = torch.randn(batch_size, obs_dim)
actions = torch.randn(batch_size, act_dim)
target_q = torch.randn(batch_size, 1)

# Optional weights tensor for the loss function
wei = torch.ones(batch_size, 1)

ARGS = {
    "obs_dim": obs_dim,
    "act_dim": act_dim,
    "hidden_dim": hidden_dim,
    "N": N,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type BatchQNet — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a BatchQNet output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Buffer Ran
ran on a synthesized input · constructed an object; no result was computed
garyzyr001/rethinking-airl/airl/algo/sac.py
pointer only (licence: NONE) · get_code("94573a894a42ebff")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, garyzyr001/rethinking-airl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

ConcatStateAction Ran
ran on a synthesized input
yhisaki/average-reward-drl/average_reward_drl/algorithms/sac.py
pointer only (licence: NONE) · get_code("c10734f04f231cb4")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, yhisaki/average-reward-drl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Critic Ran
ran on a synthesized input · constructed an object; no result was computed
MatthieuSarkis/Portfolio-Optimization-and-Goal-Based-Investment-with-Reinforcement-Learning/src/agents.py
code served (permissive licence) · get_code("7c4433b21cf61a2c")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Critic: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
import tempfile
import os

torch.manual_seed(0)

# Create a temporary directory for checkpoints
temp_dir = tempfile.mkdtemp()

# Define arguments for Critic constructor
ARGS = {
    'lr_Q': 0.001,
    'action_space_dimension': 3,
    'input_shape': (4,),
    'layer_neurons': 8,
    'network_name': 'test_critic',
    'checkpoint_directory_networks': temp_dir,
    'device': 'cpu'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Critic — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Critic output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Critic Ran
ran on a synthesized input · constructed an object; no result was computed
baturaysaglam/la3p/Code/SAC/SAC.py
code served (permissive licence) · get_code("f8ead4c2c205ecd8")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Critic: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    "num_inputs": 4,
    "num_actions": 3,
    "hidden_dim": 8,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Critic — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Critic output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Critic Ran
ran on a synthesized input · constructed an object; no result was computed
BY571/Soft-Actor-Critic-and-Extensions/files/Agent.py
code served (permissive licence) · get_code("8f0377d2cad944f3")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Critic: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    "state_size": 4,
    "action_size": 3,
    "seed": 42,
    "device": "cpu",
    "hidden_size": 32,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Critic — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Critic output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Critic Ran
ran on a synthesized input · constructed an object; no result was computed
SaminYeasar/off_policy_ac/SAC/SAC.py
pointer only (licence: NONE) · get_code("a0a4c6c892e497cd")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, SaminYeasar/off_policy_ac when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

DeepActor Ran
ran on a synthesized input · constructed an object; no result was computed
BY571/Soft-Actor-Critic-and-Extensions/files/Agent.py
code served (permissive licence) · get_code("9ee95d5b4fc2dd44")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, DeepActor: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
import numpy as np

torch.manual_seed(0)

# Define the parameters for DeepActor
state_size = 4
action_size = 3
seed = 42
device = 'cpu'
hidden_size = 32
init_w = 0.003
log_std_min = -20
log_std_max = 2

ARGS = {
    'state_size': state_size,
    'action_size': action_size,
    'seed': seed,
    'device': device,
    'hidden_size': hidden_size,
    'init_w': init_w,
    'log_std_min': log_std_min,
    'log_std_max': log_std_max,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type DeepActor — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a DeepActor output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
DeepCritic Ran
ran on a synthesized input · constructed an object; no result was computed
BY571/Soft-Actor-Critic-and-Extensions/files/Agent.py
code served (permissive licence) · get_code("a57bb45ab6a0bc51")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, DeepCritic: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
import numpy as np

torch.manual_seed(0)

# Define parameters for DeepCritic
state_size = 4
action_size = 3
seed = 42
device = torch.device('cpu')
hidden_size = 32

ARGS = {
    'state_size': state_size,
    'action_size': action_size,
    'seed': seed,
    'device': device,
    'hidden_size': hidden_size
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type DeepCritic — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a DeepCritic output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
DeepIQN Ran
ran on a synthesized input · constructed an object; no result was computed
BY571/Soft-Actor-Critic-and-Extensions/files/Agent.py
code served (permissive licence) · get_code("693c33a1aa6474c6")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, DeepIQN: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    "state_size": 4,
    "action_size": 3,
    "layer_size": 5,
    "seed": 42,
    "N": 32,
    "device": "cpu"
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type DeepIQN — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a DeepIQN output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
DeterministicPolicy Ran
ran on a synthesized input · constructed an object; no result was computed
baturaysaglam/la3p/Code/SAC/SAC.py
code served (permissive licence) · get_code("7553f61c31debefb")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, DeterministicPolicy: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

class MockActionSpace:
    def __init__(self):
        self.low = torch.tensor([-1.0, -1.0, -1.0])
        self.high = torch.tensor([1.0, 1.0, 1.0])

ARGS = {
    "num_inputs": 4,
    "num_actions": 3,
    "hidden_dim": 8,
    "action_space": MockActionSpace()
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type DeterministicPolicy — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a DeterministicPolicy output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
DeterministicPolicy Ran
ran on a synthesized input · constructed an object; no result was computed
h-aboutalebi/SparceReward/engine/algorithms/SAC/sac.py
pointer only (licence: NONE) · get_code("3faeea6e22edd34d")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, h-aboutalebi/SparceReward when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

DeterministicPolicy Ran
ran on a synthesized input · constructed an object; no result was computed
MarsEleven/car_racer_RL/sac/sac.py
pointer only (licence: NONE) · get_code("d4dfbbf39952f704")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, MarsEleven/car_racer_RL when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

GaussianPolicy Ran
ran on a synthesized input · constructed an object; no result was computed
Rafael1s/Deep-Reinforcement-Learning-Udacity/Ant-PyBulletEnv-Soft-Actor-Critic/sac_agent.py
pointer only (licence: NONE) · get_code("bf0d89394b22e1b6")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, Rafael1s/Deep-Reinforcement-Learning-Udacity when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

GaussianPolicy Ran
ran on a synthesized input · constructed an object; no result was computed
baturaysaglam/la3p/Code/SAC/SAC.py
code served (permissive licence) · get_code("152bc1e449547d23")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, GaussianPolicy: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
import numpy as np

torch.manual_seed(0)

# Create a simple action space mock
class ActionSpace:
    def __init__(self, low, high):
        self.low = low
        self.high = high

# Define parameters for GaussianPolicy
num_inputs = 4
num_actions = 3
hidden_dim = 8
action_space = ActionSpace(
    low=np.array([-1.0, -1.0, -1.0]),
    high=np.array([1.0, 1.0, 1.0])
)

ARGS = {
    "num_inputs": num_inputs,
    "num_actions": num_actions,
    "hidden_dim": hidden_dim,
    "action_space": action_space
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type GaussianPolicy — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a GaussianPolicy output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
GaussianPolicy Ran
ran on a synthesized input · constructed an object; no result was computed
ku2482/discor.pytorch/discor/algorithm/sac.py
code served (permissive licence) · get_code("8761dc745b44dc04")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, GaussianPolicy: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn

torch.manual_seed(0)

ARGS = {
    "state_dim": 4,
    "action_dim": 3,
    "hidden_units": [256, 256]
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type GaussianPolicy — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a GaussianPolicy output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
GaussianPolicy Ran
ran on a synthesized input · constructed an object; no result was computed
h-aboutalebi/SparceReward/engine/algorithms/SAC/sac.py
pointer only (licence: NONE) · get_code("8ee15fddb3528e86")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, h-aboutalebi/SparceReward when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

GaussianPolicy Ran
ran on a synthesized input · constructed an object; no result was computed
roythuly/obac/model/algo.py
pointer only (licence: NONE) · get_code("da8ebd6e9ea11fca")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, roythuly/obac when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

GaussianPolicy Ran
ran on a synthesized input · constructed an object; no result was computed
sunfex/weighted-sac/sac.py
code served (permissive licence) · get_code("5722a85a4a153169")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, GaussianPolicy: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
import numpy as np

torch.manual_seed(0)

# Create a simple action space mock object
class ActionSpace:
    def __init__(self, low, high):
        self.low = low
        self.high = high

# Define parameters for GaussianPolicy
num_inputs = 4
num_actions = 3
hidden_dim = 8
action_space = ActionSpace(
    low=np.array([-1.0, -1.0, -1.0]),
    high=np.array([1.0, 1.0, 1.0])
)

ARGS = {
    "num_inputs": num_inputs,
    "num_actions": num_actions,
    "hidden_dim": hidden_dim,
    "action_space": action_space
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type GaussianPolicy — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a GaussianPolicy output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
GaussianPolicy Ran
ran on a synthesized input · constructed an object; no result was computed
MarsEleven/car_racer_RL/sac/sac.py
pointer only (licence: NONE) · get_code("57c7562fda9bbab3")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, MarsEleven/car_racer_RL when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

IQN Ran
ran on a synthesized input · constructed an object; no result was computed
BY571/Soft-Actor-Critic-and-Extensions/files/Agent.py
code served (permissive licence) · get_code("94a135edcb95baf2")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, IQN: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
import torch.nn as nn

torch.manual_seed(0)

# Set device to CPU only
device = 'cpu'

# Define parameters for IQN
state_size = 4
action_size = 3
layer_size = 64
seed = 42
N = 32

ARGS = {
    'state_size': state_size,
    'action_size': action_size,
    'layer_size': layer_size,
    'seed': seed,
    'N': N,
    'device': device,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type IQN — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a IQN output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
Logger Ran
ran on a synthesized input · constructed an object; no result was computed
yhisaki/average-reward-drl/average_reward_drl/algorithms/sac.py
pointer only (licence: NONE) · get_code("4aec8353051fa16d")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, yhisaki/average-reward-drl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

MLPActorCritic Ran
ran on a synthesized input · constructed an object; no result was computed
tmjeong1103/RL_with_RAY/SAC_RAY/model.py
pointer only (licence: NONE) · get_code("a830c8ece63f9c57")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, tmjeong1103/RL_with_RAY when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

MLPGaussianPolicy Ran
ran on a synthesized input · constructed an object; no result was computed
tmjeong1103/RL_with_RAY/SAC_RAY/model.py
pointer only (licence: NONE) · get_code("c02fc6e11c3ba5cf")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, tmjeong1103/RL_with_RAY when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

MLPQFunction Ran
ran on a synthesized input · constructed an object; no result was computed
tmjeong1103/RL_with_RAY/SAC_RAY/model.py
pointer only (licence: NONE) · get_code("14211bf5685457cf")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, tmjeong1103/RL_with_RAY when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

MultiLinear Ran
ran on a synthesized input · constructed an object; no result was computed
yhisaki/average-reward-drl/average_reward_drl/algorithms/sac.py
pointer only (licence: NONE) · get_code("5d3a98fbf8446e0c")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, yhisaki/average-reward-drl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Network Ran
ran on a synthesized input · constructed an object; no result was computed
MatthieuSarkis/Portfolio-Optimization-and-Goal-Based-Investment-with-Reinforcement-Learning/src/agents.py
code served (permissive licence) · get_code("de764e9177884a12")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, Network: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. The code mentions cuda, and the harness aliases cuda to the CPU for every sample; whether this run needed that is not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import os
import tempfile

torch.manual_seed(0)

# Create a temporary directory for checkpoints
temp_dir = tempfile.mkdtemp()

ARGS = {
    'input_shape': (4,),
    'layer_neurons': 64,
    'network_name': 'test_network',
    'checkpoint_directory_networks': temp_dir,
    'device': 'cpu'
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Network — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a Network output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
PolicyNetwork Ran
ran on a synthesized input · constructed an object; no result was computed
quantumiracle/Popular-RL-Algorithms/sac_v2.py
code served (permissive licence) · get_code("540316788d2c30c7")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, PolicyNetwork: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
import numpy as np

torch.manual_seed(0)

class PolicyNetwork(nn.Module):

    def __init__(self, num_inputs, num_actions, hidden_size, action_range=1.0, init_w=0.003, log_std_min=-20, log_std_max=2):
        super(PolicyNetwork, self).__init__()
        self.log_std_min = log_std_min
        self.log_std_max = log_std_max
        self.linear1 = nn.Linear(num_inputs, hidden_size)
        self.linear2 = nn.Linear(hidden_size, hidden_size)
        self.linear3 = nn.Linear(hidden_size, hidden_size)
        self.linear4 = nn.Linear(hidden_size, hidden_size)
        self.mean_linear = nn.Linear(hidden_size, num_actions)
        self.mean_linear.weight.data.uniform_(-init_w, init_w)
        self.mean_linear.bias.data.uniform_(-init_w, init_w)
        self.log_std_linear = nn.Linear(hidden_size, num_actions)
        self.log_std_linear.weight.data.uniform_(-init_w, init_w)
        self.log_std_linear.bias.data.uniform_(-init_w, init_w)
        self.action_range = action_range
        self.num_actions = num_actions

    def forward(self, state):
        x = torch.nn.functional.relu(self.linear1(state))
        x = torch.nn.functional.relu(self.linear2(x))
        x = torch.nn.functional.relu(self.linear3(x))
        x = torch.nn.functional.relu(self.linear4(x))
        mean = self.mean_linear(x)
        log_std = self.log_std_linear(x)
        log_std = torch.clamp(log_std, self.log_std_min, self.log_std_max)
        return (mean, log_std)

    def evaluate(self, state, epsilon=1e-06):
        mean, log_std = self.forward(state)
        std = log_std.exp()
        normal = torch.distributions.Normal(0, 1)
        z = normal.sample(mean.shape)
        action_0 = torch.tanh(mean + std * z)
        action = self.action_range * action_0
        log_prob = torch.distributions.Normal(mean, std).log_prob(mean + std * z) - torch.log(1.0 - action_0.pow(2) + epsilon) - np.log(self.action_range)
        log_prob = log_prob.sum(dim=1, keepdim=True)
        return (action, log_prob, z, mean, log_std)

    def get_action(self, state, deterministic):
        state = torch.FloatTensor(state).unsqueeze(0)
        mean, log_std = self.forward(state)
        std = log_std.exp()
        normal = torch.distributions.Normal(0, 1)
        z = normal.sample(mean.shape)
        action = self.action_range * torch.tanh(mean + std * z)
        action = self.action_range * torch.tanh(mean).detach().numpy()[0] if deterministic else action.detach().numpy()[0]
        return action

    def sample_action(self):
        a = torch.FloatTensor(self.num_actions).uniform_(-1, 1)
        return self.action_range * a.numpy()

ARGS = {
    'num_inputs': 4,
    'num_actions': 3,
    'hidden_size': 8,
    'action_range': 1.0,
    'init_w': 0.003,
    'log_std_min': -20,
    'log_std_max': 2
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type PolicyNetwork — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a PolicyNetwork output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
PolicyNetwork Ran
ran on a synthesized input
ajaysub110/rl-pytorch/sac/models/soft_actor_critic.py
pointer only (licence: NONE) · get_code("ec7050db6c32a43a")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ajaysub110/rl-pytorch when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

PrioritizedReplay Ran
ran on a synthesized input · constructed an object; no result was computed
BY571/Soft-Actor-Critic-and-Extensions/files/Agent.py
code served (permissive licence) · get_code("91bcdaabd64f8eee")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, PrioritizedReplay: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np

torch.manual_seed(0)
np.random.seed(42)

ARGS = {
    "capacity": 1000,
    "batch_size": 32,
    "device": torch.device("cpu"),
    "seed": 42,
    "gamma": 0.99,
    "alpha": 0.6,
    "beta_start": 0.4,
    "beta_frames": 100000,
    "n_step": 3,
    "parallel_env": 4,
    "ere": False,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type PrioritizedReplay — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a PrioritizedReplay output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
QNetwork Ran
ran on a synthesized input · constructed an object; no result was computed
Rafael1s/Deep-Reinforcement-Learning-Udacity/Ant-PyBulletEnv-Soft-Actor-Critic/sac_agent.py
pointer only (licence: NONE) · get_code("f37748964b0cd7da")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, Rafael1s/Deep-Reinforcement-Learning-Udacity when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

QNetwork Ran
ran on a synthesized input · constructed an object; no result was computed
thomashirtz/soft-actor-critic/soft_actor_critic/agent.py
code served (permissive licence) · get_code("b7030d8d56cd639a")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, QNetwork: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from typing import List

torch.manual_seed(0)

ARGS = {
    "input_dims": 4,
    "num_actions": 3,
    "hidden_units": [16, 8]
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type QNetwork — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a QNetwork output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
QNetwork Ran
ran on a synthesized input · constructed an object; no result was computed
roythuly/obac/model/algo.py
pointer only (licence: NONE) · get_code("0d80189fd77d3aa6")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, roythuly/obac when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

QNetwork Ran
ran on a synthesized input · constructed an object; no result was computed
MarsEleven/car_racer_RL/sac/sac.py
pointer only (licence: NONE) · get_code("e3aca820c0049604")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, MarsEleven/car_racer_RL when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

QNetwork Ran
ran on a synthesized input
ajaysub110/rl-pytorch/sac/models/soft_actor_critic.py
pointer only (licence: NONE) · get_code("e182a5c8f1d1e16f")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ajaysub110/rl-pytorch when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

ReplayBuffer Ran
ran on a synthesized input · constructed an object; no result was computed
MatthieuSarkis/Portfolio-Optimization-and-Goal-Based-Investment-with-Reinforcement-Learning/src/agents.py
code served (permissive licence) · get_code("ac5ecca5b7705f8a")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ReplayBuffer: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np
from typing import Tuple

# Define ARGS dict with all parameters for ReplayBuffer.__init__
ARGS = {
    "size": 64,
    "input_shape": (4, 3),
    "action_space_dimension": 2
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ReplayBuffer — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ReplayBuffer output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ReplayBuffer Ran
ran on a synthesized input · constructed an object; no result was computed
kdally/fault-tolerant-flight-control-drl/fault_tolerant_flight_control_drl/agent/sac.py
code served (permissive licence) · get_code("8119b0afbe2d940e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ReplayBuffer: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np
import random

random.seed(0)
np.random.seed(0)

ARGS = {
    "size": 100
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ReplayBuffer — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ReplayBuffer output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ReplayBuffer Ran
ran on a synthesized input · constructed an object; no result was computed
BY571/Soft-Actor-Critic-and-Extensions/files/Agent.py
code served (permissive licence) · get_code("a6c39d1200737d3f")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ReplayBuffer: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import random
import torch
import numpy as np
from collections import namedtuple, deque

torch.manual_seed(0)
np.random.seed(0)
random.seed(0)

# Construct valid arguments for ReplayBuffer
ARGS = {
    'buffer_size': 100,
    'batch_size': 4,
    'device': torch.device('cpu'),
    'seed': 42,
    'gamma': 0.99,
    'n_step': 2,
    'parallel_env': 2,
    'ere': False
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ReplayBuffer — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ReplayBuffer output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ReplayBuffer Ran
ran on a synthesized input · constructed an object; no result was computed
yhisaki/average-reward-drl/average_reward_drl/algorithms/sac.py
pointer only (licence: NONE) · get_code("e9664791458e5ade")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, yhisaki/average-reward-drl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

ReplayBuffer Ran
ran on a synthesized input · constructed an object; no result was computed
quantumiracle/Popular-RL-Algorithms/sac_v2.py
code served (permissive licence) · get_code("70249df825c62d7e")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ReplayBuffer: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import random
import numpy as np
import torch

torch.manual_seed(0)

# The ReplayBuffer class takes only one parameter: capacity
# The class stores tuples of (state, action, reward, next_state, done)
# and samples batches from them.
# We need to set up a buffer with some sample data so it can be sampled from.

# Create a ReplayBuffer instance and populate it with sample transitions
capacity = 500

buffer = type('ReplayBuffer', (), {
    'capacity': capacity,
    'buffer': [],
    'position': 0,
    'push': lambda self, state, action, reward, next_state, done: (
        self.buffer.append(None) if len(self.buffer) < self.capacity else None,
        self.buffer.__setitem__(self.position, (state, action, reward, next_state, done)),
        setattr(self, 'position', int((self.position + 1) % self.capacity))
    ),
})()

# Populate buffer with sample transitions
# state: shape (4,), action: scalar, reward: scalar, next_state: shape (4,), done: bool
for i in range(350):
    state = np.random.randn(4).astype(np.float32)
    action = np.random.randint(0, 2)
    reward = np.random.randn()
    next_state = np.random.randn(4).astype(np.float32)
    done = bool(np.random.rand() > 0.7)
    buffer.push(state, action, reward, next_state, done)

ARGS = {
    'capacity': capacity
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ReplayBuffer — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ReplayBuffer output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
ReplayBuffer Ran
ran on a synthesized input · constructed an object; no result was computed
thomashirtz/soft-actor-critic/soft_actor_critic/agent.py
code served (permissive licence) · get_code("0020d85f4dbfb10b")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, ReplayBuffer: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import numpy as np

# ReplayBuffer parameters
max_size = 1000
input_shape = 4  # state dimension
num_actions = 3  # number of possible actions

ARGS = {
    "max_size": max_size,
    "input_shape": input_shape,
    "num_actions": num_actions,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
numpy — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type ReplayBuffer — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a ReplayBuffer output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
SAC Ran
ran on a synthesized input · constructed an object; no result was computed
ShawK91/Evolutionary-Reinforcement-Learning/algos/sac.py
pointer only (licence: NONE) · get_code("217cd7f6f86864da")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, ShawK91/Evolutionary-Reinforcement-Learning when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

SAC Ran
ran on a synthesized input · constructed an object; no result was computed
SaminYeasar/off_policy_ac/SAC/SAC.py
pointer only (licence: NONE) · get_code("43d63d9b3b6240a1")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, SaminYeasar/off_policy_ac when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

SACActor Ran
ran on a synthesized input · constructed an object; no result was computed
kushagra06/SAC/model.py
pointer only (licence: NONE) · get_code("e5d43066162708ff")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, kushagra06/SAC when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

ScalarHolder Ran
ran on a synthesized input · constructed an object; no result was computed
yhisaki/average-reward-drl/average_reward_drl/algorithms/sac.py
pointer only (licence: NONE) · get_code("33ec92b652664841")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, yhisaki/average-reward-drl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

SerializedBuffer Ran
ran on a synthesized input · constructed an object; no result was computed
garyzyr001/rethinking-airl/airl/algo/sac.py
pointer only (licence: NONE) · get_code("c74fba5d30037b69")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, garyzyr001/rethinking-airl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

SerializedBuffer Ran
ran on a synthesized input
ku2482/gail-airl-ppo.pytorch/gail_airl_ppo/algo/sac.py
code served (permissive licence) · get_code("eb9bc09b3ca641cb")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned tuple. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import numpy as np
import tempfile
import os

# Create a temporary file with serialized buffer data
temp_dir = tempfile.mkdtemp()
temp_path = os.path.join(temp_dir, "buffer.pt")

# Create sample data
batch_size = 8
state_dim = 4
action_dim = 2

sample_data = {
    'state': torch.randn(batch_size, state_dim),
    'action': torch.randn(batch_size, action_dim),
    'reward': torch.randn(batch_size, 1),
    'done': torch.randint(0, 2, (batch_size, 1)).float(),
    'next_state': torch.randn(batch_size, state_dim),
}

torch.save(sample_data, temp_path)

INIT_ARGS = {
    'path': temp_path,
    'device': 'cpu',
}

ARGS = {
    'batch_size': 3,
}

METHOD = 'sample'
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type tuple — a tuple output has no array shape of its own; none is stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)
SoftActor Ran
ran on a synthesized input · constructed an object; no result was computed
Kaixhin/imitation-learning/models.py
code served (permissive licence) · get_code("b473c449b3da39cc")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, SoftActor: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn
from omegaconf import DictConfig

torch.manual_seed(0)

# Create model_cfg as a DictConfig
model_cfg = DictConfig({
    'hidden_size': 16,
    'depth': 2,
    'activation': 'relu',
    'input_dropout': 0.1,
    'dropout': 0.05
})

# Define ARGS mapping all parameters of SoftActor.__init__
ARGS = {
    'state_size': 5,
    'action_size': 3,
    'model_cfg': model_cfg
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
omegaconf, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type SoftActor — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a SoftActor output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
SoftQNetwork Ran
ran on a synthesized input · constructed an object; no result was computed
quantumiracle/Popular-RL-Algorithms/sac_v2.py
code served (permissive licence) · get_code("0688d4da92feee38")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, SoftQNetwork: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
import torch.nn as nn

torch.manual_seed(0)

ARGS = {
    "num_inputs": 4,
    "num_actions": 3,
    "hidden_size": 8,
    "init_w": 0.003,
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type SoftQNetwork — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a SoftQNetwork output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
SquashedDiagonalGaussianHead Ran
ran on a synthesized input · constructed an object; no result was computed
yhisaki/average-reward-drl/average_reward_drl/algorithms/sac.py
pointer only (licence: NONE) · get_code("02a45597a277afab")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, yhisaki/average-reward-drl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

StateActionFunction Ran
ran on a synthesized input · constructed an object; no result was computed
ku2482/discor.pytorch/discor/algorithm/sac.py
code served (permissive licence) · get_code("b1e066bc1ff4b61b")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API) and returned an instance of its own class, StateActionFunction: the run constructed the object and did not call it, so no result was computed. The image it ran in was not recorded. If your copy returns the same type on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn

torch.manual_seed(0)

ARGS = {
    "state_dim": 4,
    "action_dim": 3,
    "hidden_units": [256, 256]
}
Setup block drafted by
a language model, Claude Haiku 4.5, prompt python_fixture_v1, through the batch API
Imports
torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type StateActionFunction — the run constructed an instance of its own class and did not call it; no shape is stated — the instrument coerces any output to an array; for a StateActionFunction output the recorded ndim 0 and dtype family float describe that coercion, not the output, and are not stated
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: none by the harness; any seed is in the setup block
StateDependentPolicy Ran
ran on a synthesized input · constructed an object; no result was computed
garyzyr001/rethinking-airl/airl/algo/sac.py
pointer only (licence: NONE) · get_code("f0f9d0b00b3e9632")
How it ran

The body of this sample is withheld: GitHub detected no licence in this paper's repository, garyzyr001/rethinking-airl when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

StateDependentPolicy Ran
ran on a synthesized input
ku2482/gail-airl-ppo.pytorch/gail_airl_ppo/algo/sac.py
code served (permissive licence) · get_code("77baa7a1a168fc24")
How it ran

Ran in Syntology's sandbox on this synthesized input (drafted by a language model, class construct-then-call prompt, model not recorded) and returned Tensor, ndim 2, dtype family float; no output shape is stated. The image it ran in was not recorded. If your copy returns the same type and ndim on these inputs, it matched what we observed; nothing here checks that. It does NOT mean the output is correct: nothing has compared it to the paper.

import torch
from torch import nn
import numpy as np

INIT_ARGS = {
    "state_shape": (4,),
    "action_shape": (3,),
    "hidden_units": (8, 8),
    "hidden_activation": nn.ReLU(inplace=True)
}

# Create a small batch of states (batch_size=2, state_dim=4)
states = torch.randn(2, 4)

ARGS = {
    "states": states
}
Setup block drafted by
a language model, class construct-then-call prompt, model not recorded
Imports
numpy, torch — import names as written, not package names to install; a name the repository itself provides is not a package
Returned
type Tensor, ndim 2, dtype family float — no output shape is stated: observe_signature.py observed this class with its constructor sizes canonicalised where it could, which the node does not record, so its shape is not stated as this setup block's
dtype family
int means an integer dtype; float means any other dtype the instrument saw, bool and complex included (fixture_check.py records only these two families)
Environment
image not recorded; re-verify against syntology-harvest-sandbox:v5-deps; not recorded for this sample: the harness installs a cuda-to-cpu shim (cuda_shim.py) before every sample and no run records whether a sample needed it; seed: torch and numpy seeded to 0 by the harness immediately before the class was constructed from INIT_ARGS (fixture_check.py)

Showing the first 60. An agent gets every one from get_harvested_code_for_paper("1801.01290").

Also present in other papers' code

Of 148 samples lifted from this paper's linked repositories, 148 carry a function identity and 0 do not. Of the 139 distinct functions with an identity, 30 are also present in other papers whose code we hold (6 with at least one paper dated earlier, 0 the same month, 26 later) and 109 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
weights_init_
brick of this paper's assembly: helper
10 other papers
10 later · 10 only as an assembly brick
2006.07185 (2020-06)
GROUNDING LANGUAGE TO AUTONOMOUSLY-ACQUIRED SKILLS VIA GOAL GENERATION
2007.01932 (2020-07)
Meta-SAC: Auto-tune the Entropy Temperature of Soft Actor-Critic via Metagradient
2010.08830 (2020-10)
MESA: Boost Ensemble Imbalanced Learning with MEta-SAmpler
and 7 more not named here
mlp
brick of this paper's assembly: helper
5 other papers
5 later
1802.09477 (2018-02)
Addressing Function Approximation Error in Actor-Critic Methods
1812.02690 (2018-12)
Provably Efficient Maximum Entropy Exploration
1910.07224 (2019-10)
Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments
and 2 more not named here
hard_update
brick of this paper's assembly: helper, update_rule
4 other papers
1 dated earlier, 3 later · 3 only as an assembly brick
1706.02275 (2017-06)
Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
2007.01932 (2020-07)
Meta-SAC: Auto-tune the Entropy Temperature of Soft Actor-Critic via Metagradient
2010.08830 (2020-10)
MESA: Boost Ensemble Imbalanced Learning with MEta-SAmpler
and 1 more not named here
soft_update
brick of this paper's assembly: helper, update_rule
4 other papers
1 dated earlier, 3 later · 3 only as an assembly brick
1706.02275 (2017-06)
Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
2007.01932 (2020-07)
Meta-SAC: Auto-tune the Entropy Temperature of Soft Actor-Critic via Metagradient
2010.08830 (2020-10)
MESA: Boost Ensemble Imbalanced Learning with MEta-SAmpler
and 1 more not named here
AlgorithmBase
brick of this paper's assembly: helper
2 other papers
2 later · 1 only as an assembly brick
2305.12239 (2023-05)
Off-Policy Average Reward Actor-Critic with Deterministic Policy Search
2408.01972 (2024-08)
RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning
Batch
brick of this paper's assembly: helper
2 other papers
2 later · 2 only as an assembly brick
2305.12239 (2023-05)
Off-Policy Average Reward Actor-Critic with Deterministic Policy Search
2408.01972 (2024-08)
RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning
ConcatStateAction
brick of this paper's assembly: helper
2 other papers
2 later · 2 only as an assembly brick
2305.12239 (2023-05)
Off-Policy Average Reward Actor-Critic with Deterministic Policy Search
2408.01972 (2024-08)
RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning
Logger
brick of this paper's assembly: helper
2 other papers
2 later · 1 only as an assembly brick
2305.12239 (2023-05)
Off-Policy Average Reward Actor-Critic with Deterministic Policy Search
2408.01972 (2024-08)
RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning
MultiLinear
brick of this paper's assembly: helper
2 other papers
2 later · 2 only as an assembly brick
2305.12239 (2023-05)
Off-Policy Average Reward Actor-Critic with Deterministic Policy Search
2408.01972 (2024-08)
RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning
ReplayBuffer
brick of this paper's assembly: helper
2 other papers
2 later · 1 only as an assembly brick
2305.12239 (2023-05)
Off-Policy Average Reward Actor-Critic with Deterministic Policy Search
2408.01972 (2024-08)
RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning
ScalarHolder
brick of this paper's assembly: helper
2 other papers
2 later · 2 only as an assembly brick
2305.12239 (2023-05)
Off-Policy Average Reward Actor-Critic with Deterministic Policy Search
2408.01972 (2024-08)
RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning
convert_to_tensor
brick of this paper's assembly: helper
2 other papers
2 later · 2 only as an assembly brick
2305.12239 (2023-05)
Off-Policy Average Reward Actor-Critic with Deterministic Policy Search
2408.01972 (2024-08)
RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning
group_by_keys
brick of this paper's assembly: helper
2 other papers
2 later · 1 only as an assembly brick
2305.12239 (2023-05)
Off-Policy Average Reward Actor-Critic with Deterministic Policy Search
2408.01972 (2024-08)
RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning
polyak_update
brick of this paper's assembly: helper
2 other papers
2 later · 2 only as an assembly brick
2305.12239 (2023-05)
Off-Policy Average Reward Actor-Critic with Deterministic Policy Search
2408.01972 (2024-08)
RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning
target_update
brick of this paper's assembly: helper
2 other papers
2 later · 1 only as an assembly brick
2210.04251 (2022-10)
State Advantage Weighting for Offline RL
2304.10466 (2023-04)
Efficient Deep Reinforcement Learning Requires Regulating Overfitting
AbstractAgent
brick of this paper's assembly: helper
1 other paper
1 dated earlier · 1 only as an assembly brick
1509.02971 (2015-09)
Continuous control with deep reinforcement learning
Algorithm
brick of this paper's assembly: helper
1 other paper
1 dated earlier
1707.06347 (2017-07)
Proximal Policy Optimization Algorithms
BaseNetwork
brick of this paper's assembly: helper
1 other paper
1 later
2007.14430 (2020-07)
Munchausen Reinforcement Learning
DeterministicPolicy
brick of this paper's assembly: model
1 other paper
1 later
2010.08830 (2020-10)
MESA: Boost Ensemble Imbalanced Learning with MEta-SAmpler
Model
brick of this paper's assembly: helper
1 other paper
1 later · 1 only as an assembly brick
2304.10466 (2023-04)
Efficient Deep Reinforcement Learning Requires Regulating Overfitting

Shown: the 20 functions present in the most other papers. 10 more shared functions are not listed here.

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

More are linked than shown here.

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Model-free deep reinforcement learning (RL) algorithms have been demonstrated on a range of challenging decision making and control tasks. However, these methods typically suffer from two major challenges: very high sample complexity and brittle convergence properties, which necessitate meticulous hyperparameter tuning. Both of these challenges severely limit the applicability of such methods to complex, real-world domains. In this paper, we propose soft actor-critic, an offpolicy actor-critic deep RL algorithm based on the maximum entropy reinforcement learning framework. In this framework, the actor aims to maximize expected reward while also maximizing entropy. That is, to succeed at the task while acting as randomly as possible. Prior deep RL methods based on this framework have been formulated as Q-learning methods. By combining off-policy updates with a stable stochastic actor-critic formulation, our method achieves state-of-the-art performance on a range of continuous control benchmark tasks, outperforming prior on-policy and off-policy methods. Furthermore, we demonstrate that, in contrast to other off-policy algorithms, our approach is very stable, achieving very similar performance across different random seeds. We explore how to design an efficient and stable modelfree deep RL algorithm for continuous state and action spaces. To that end, we draw on the maximum entropy framework, which augments the standard maximum reward reinforcement learning objective with an entropy maximization term (

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("1801.01290")
get_code_for_paper("1801.01290")
have("1801.01290")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/1801.01290.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/1801.01290.svg)](https://syntology.ai/paper/1801.01290)

Connect an agent — have() is free.