We lifted 5 functions out of this paper's own repositories and ran 2 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| copy not recorded | — | 1 of 1 |
| jangirrishabh/Overcoming-exploration-from-demos | reimplementation | 1 of 1 |
| kaixindelele/DRLib | pwc_unofficial | 0 of 3 |
| Function | Status | Where it lives |
|---|---|---|
| dims_to_shapes | Ran | this paper's copy was not recorded; identical code first harvested from yangrui2015/goat pointer only · get_code("a68f6550efbfd9a5") |
| make_sample_her_transitions | Ran | jangirrishabh/Overcoming-exploration-from-demos/her.py code served (permissive licence) · get_code("e0785843af667cdd") |
| add_noise_obs2state | Not yet run | kaixindelele/DRLib/D2SSR/d2ssr_train_torch.py code served (permissive licence) · get_code("6874fb8652a4c8a3") |
| obs2state | Not yet run | kaixindelele/DRLib/train_tf1.py code served (permissive licence) · get_code("6a2c71b97236af90") |
| obs2state | Not yet run | kaixindelele/DRLib/D2SSR/d2ssr_train_torch.py code served (permissive licence) · get_code("bd60c9cb106e111f") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Exploration in environments with sparse rewards has been a persistent problem in reinforcement learning (RL). Many tasks are natural to specify with a sparse reward, and manually shaping a reward function can result in suboptimal performance. However, finding a non-zero reward is exponentially more difficult with increasing task horizon or action dimensionality. This puts many real-world tasks out of practical reach of RL methods. In this work, we use demonstrations to overcome the exploration problem and successfully learn to perform long-horizon, multi-step robotics tasks with continuous control such as stacking blocks with a robot arm. Our method, which builds on top of Deep Deterministic Policy Gradients and Hindsight Experience Replay, provides an order of magnitude of speedup over RL on simulated robotics tasks. It is simple to implement and makes only the additional assumption that we can collect a small set of demonstrations. Furthermore, our method is able to solve tasks not solvable by either RL or behavior cloning alone, and often ends up outperforming the demonstrator policy.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("1709.10089")
get_code_for_paper("1709.10089")
have("1709.10089")
Connect an agent — have() is free.