SYNTOLOGY HomeExplorerAtlasCodeMethodologyAboutDevelopersFeedPricing
Paper · 2206.03023 · NeurIPS · 2022

Offline Goal-Conditioned Reinforcement Learning via f -Advantage Regression

Jason Yecheng, Osbert Jayaraman, Bastani, Jason Ma, Dinesh Yan

arXiv · PDF · Open in the Atlas

Code that ran

5 of 5 samples (functions and classes) lifted from this paper's linked repositories executed on a synthesized input in Syntology's jail (0 of them constructed an object rather than computing a result; 1 honoured the contract we drafted, 1 violated it and 0 have no contract verdict; 3 failures of our instrument; 0 no recorded run), measured 2026-11-22; repository links from the Papers with Code archive attributed under CC BY-SA, no affiliation; bodies inline only under a permissive licence as GitHub detects it. Not a reproduction of the paper's results.

"Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results. This record, dated, as JSON: /api/ran/2206.03023.json · badge: /api/ran/2206.03023.svg

Each repository's role is a label: a classifier's output recorded on the link, or, where no role is recorded, what the Papers with Code archive marked. The line under it names its basis. Neither says who wrote the code.

RepositoryRoleRan
jasonma2016/gofar canonical (classifier label)
classifier basis not recorded on the link
5 ran · 0 no recorded run
FunctionStatusWhere it lives
atanh Ran
ran; our contract draft was wrong, not the code
jasonma2016/gofar/rl_modules/models.py
pointer only (licence: MIT); the stored copy get_code serves, from jasonma2016/smodice, is pointer only (licence: NONE) · get_code("cf5ce98955ab4534")
How it ran

The body of this sample is withheld: this paper's link allows the body inline (MIT), but get_code serves the stored copy, first harvested from jasonma2016/smodice, and GitHub detected no licence in that repository when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

create_stats_ordered_dict Ran
ran; the contract check could not run (RAISES, DEP_MISSING or TIMEOUT)
jasonma2016/gofar/envs/env_util.py
pointer only (licence: MIT); the stored copy get_code serves, from moreanp/csro, is pointer only (licence: NONE) · get_code("f395bfda2e4813d7")
How it ran

The body of this sample is withheld: this paper's link allows the body inline (MIT), but get_code serves the stored copy, first harvested from moreanp/csro, and GitHub detected no licence in that repository when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

get_average_returns Ran
ran, honoured the contract we drafted
jasonma2016/gofar/envs/env_util.py
pointer only (licence: MIT); the stored copy get_code serves, from aviralkumar2907/CQL, is pointer only (licence: NONE) · get_code("a5d10a0b704c8db7")
How it ran

The body of this sample is withheld: this paper's link allows the body inline (MIT), but get_code serves the stored copy, first harvested from aviralkumar2907/CQL, and GitHub detected no licence in that repository when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

get_generic_path_information Ran
ran; our contract draft was wrong, not the code
jasonma2016/gofar/envs/env_util.py
pointer only (licence: MIT); the stored copy get_code serves, from moreanp/csro, is pointer only (licence: NONE) · get_code("c809703793017cd2")
How it ran

The body of this sample is withheld: this paper's link allows the body inline (MIT), but get_code serves the stored copy, first harvested from moreanp/csro, and GitHub detected no licence in that repository when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

goal_distance Ran
ran, violated the contract we drafted
jasonma2016/gofar/rl_modules/base_agent.py
pointer only (licence: MIT); the stored copy get_code serves, from sunlighted/rbsl, is pointer only (licence: NONE) · get_code("6eb59c36d3f33fe4")
How it ran

The body of this sample is withheld: this paper's link allows the body inline (MIT), but get_code serves the stored copy, first harvested from sunlighted/rbsl, and GitHub detected no licence in that repository when it was checked. The setup block that called it is withheld with it, because it is written against that body. source_path and origin say where the code lives, to read under the repository's own terms. A run is recorded.

Also present in other papers' code

Of 5 samples lifted from this paper's linked repositories, 5 carry a function identity and 0 do not. Of the 5 distinct functions with an identity, 5 are also present in other papers whose code we hold (5 with at least one paper dated earlier, 1 the same month, 5 later) and 0 are only here among the 52,977 papers whose code Syntology holds (as of 2026-09-26).

Only here means: among the 52,977 papers whose code Syntology holds (51,887 with lifted samples, 1,090 through their assemblies only), as of 2026-09-26, no other paper's lifted samples or current assembly bricks contain an identical function body after normalisation.

FunctionAlso present inEarliest-datedOthers named
get_average_returns 24 other papers
10 dated earlier, 1 the same month, 13 later
1903.08254 (2019-03)
Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables
1910.11670 (2019-10)
Contextual Imagined Goals for Self-Supervised Robotic Learning
1912.11032 (2019-12)
Towards Practical Multi-Object Manipulation using Relational Reinforcement Learning
and 21 more not named here
atanh 4 other papers
1 dated earlier, 3 later
2202.02433 (2022-02)
Versatile Offline Imitation from Observations and Examples via Regularized State-Occupancy Matching
2311.01331 (2023-11)
Offline Imitation from Observation via Primal Wasserstein State Occupancy Matching
2403.01734 (2024-03)
Offline Goal-Conditioned Reinforcement Learning for Safety-Critical Tasks with Recovery Policy
and 1 more not named here
create_stats_ordered_dict 4 other papers
2 dated earlier, 2 later
1903.08254 (2019-03)
Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables
1912.11032 (2019-12)
Towards Practical Multi-Object Manipulation using Relational Reinforcement Learning
2311.03695 (2023-11)
Context Shift Reduction for Offline Meta-Reinforcement Learning
and 1 more not named here
goal_distance 4 other papers
2 dated earlier, 2 later
1806.07851 (2018-06)
Sim-to-Real Reinforcement Learning for Deformable Object Manipulation
2104.06922 (2021-04)
Safe Continuous Control with Constrained Model-Based Policy Optimization
2402.07226 (2024-02)
Stitching Sub-trajectories with Conditional Diffusion Model for Goal-Conditioned Offline RL
and 1 more not named here
get_generic_path_information 3 other papers
1 dated earlier, 2 later
1903.08254 (2019-03)
Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables
2311.03695 (2023-11)
Context Shift Reduction for Offline Meta-Reinforcement Learning
2502.03752 (2025-02)
Self-Improving Skill Learning for Robust Skill-based Meta-Reinforcement Learning

Presence, not direction: an identical function body in two papers' repositories does not say who wrote it, which came from which, or that either paper's method rests on it. Matching is exact after normalisation, so a renamed or edited function is not matched. Dates are when papers appeared, not when code was written. Only .py files the harvester lifted are covered; notebooks are not read. A new-style arXiv id YYMM.NNNN dates a paper to the month 20YY-MM; otherwise Paper.year dates it to the year; otherwise it is undated. Dates are when the paper appeared, not when the code was written. Read 2026-11-22.

Repositories linked to this paper

Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.

Abstract

Offline goal-conditioned reinforcement learning (GCRL) promises general-purpose skill learning in the form of reaching diverse goals from purely offline datasets. We propose Goal-conditioned f -Advantage Regression (GoFAR), a novel regressionbased offline GCRL algorithm derived from a state-occupancy matching perspective; the key intuition is that the goal-reaching task can be formulated as a stateoccupancy matching problem between a dynamics-abiding imitator agent and an expert agent that directly teleports to the goal. In contrast to prior approaches, GoFAR does not require any hindsight relabeling and enjoys uninterleaved optimization for its value and policy networks. These distinct features confer GoFAR with much better offline performance and stability as well as statistical performance guarantee that is unattainable for prior methods. Furthermore, we demonstrate that GoFAR's training objectives can be re-purposed to learn an agent-independent goal-conditioned planner from purely offline source-domain data, which enables zero-shot transfer to new target domains. Through extensive experiments, we validate GoFAR's effectiveness in various problem settings and tasks, significantly outperforming prior state-of-art. Notably, on a real robotic dexterous manipulation task, while no other method makes meaningful progress, GoFAR acquires complex manipulation behavior that successfully accomplishes diverse goals. Offline Goal-Conditioned Reinforcement Learning. One core challenge of (offline) GCRL [18,43] is the sparse-reward nature of goal-reaching tasks. A popular strategy is hindsight goal relabeling [3] (or HER), which relabels off-policy transitions with future goals they have achieved instead of the original commanded goals. Existing offline GCRL algorithms [6,47] adapt HER-based online GCRL algorithms to the offline setting by incorporating additional components conducive to offline training. [6] builds on actor-critic style GCRL algorithms [3] by adding conservative Q-learning [24] as well as goal-chaining, which expands the pool of candidate relabeling goals to the entire offline dataset. Besides actor-critic methods, goal-conditioned behavior cloning, coupled with HER, has been shown to be a simple and effective method [14]. [47] improves upon [14] by incorporating discount-factor and advantage function weighting [37,38], and shows improved performance in offline GCRL. Diverging from existing literature, GoFAR does not tackle offline GCRL by adapting an online algorithm; instead, it approaches offline GCRL from a novel perspective of state-occupancy matching and derives an elegant algorithm from the dual formulation that is naturally suited for the offline setting and carries many distinct properties that make it more stable, scalable, and versatile. State-Occupancy Matching. State occupancy matching objectives have been shown to be effective for learning from observations [32,49], exploration [27], as well as matching a hand-designed target state distribution [13]; occupancy matching, in general, has been explored in the imitation learning literature [17,45,13,20]. Our strategy of estimating the state-occupancy ratio is related to DICE techniques for offline policy optimization [33,35,26,21,32]. The closest work to ours is [32], which proposed f -divergence state-occupancy matching for offline imitation learning. Our work shows that such a state-occupancy matching approach can be applied to GCRL. Algorithmically, GoFAR differs from [32] in that it does not require a separate dataset of expert demonstrations; it achieves this by modifying the discriminator, and in some settings can even be used without a discriminator, whereas a discriminator is always required by [32]. Furthermore, we derive several new algorithmic properties (Section 4.2-4.4) that are novel and particularly advantageous for offline GCRL.

For agents

The same record, over MCP at https://syntology.ai/mcp:

get_harvested_code_for_paper("2206.03023")
get_code_for_paper("2206.03023")
have("2206.03023")

The run record, dated, one paper per request, free:

curl https://syntology.ai/api/ran/2206.03023.json

A badge for a README (the split and the date, never a ratio):

[![Syntology run record](https://syntology.ai/api/ran/2206.03023.svg)](https://syntology.ai/paper/2206.03023)

Connect an agent — have() is free.