Francesco Faccio, Louis Kirsch, J Ürgen Schmidhuber
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Traditional off-policy actor-critic Reinforcement Learning (RL) algorithms learn value functions of a single target policy. However, when value functions are updated to track the learned policy, they forget potentially useful information about old policies. We introduce a class of value functions called Parameter-Based Value Functions (PBVFs) whose inputs include the policy parameters. They can generalize across different policies. PBVFs can evaluate the performance of any policy given a state, a state-action pair, or a distribution over the RL agent's initial states. First we show how PBVFs yield novel off-policy policy gradient theorems. Then we derive off-policy actor-critic algorithms based on PBVFs trained by Monte Carlo or Temporal Difference methods. We show how learned PBVFs can zeroshot learn new policies that outperform any policy seen during training. Finally our algorithms are evaluated on a selection of discrete and continuous control tasks using shallow policies and deep neural networks. Their performance is comparable to state-of-the-art methods.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2006.09226")
get_code_for_paper("2006.09226")
have("2006.09226")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2006.09226.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2006.09226)
Connect an agent — have() is free.