Sergey Levine, Clare Lyle, Angelos Filos, Yarin Gal, Gregory Farquhar, Natasha Jaques
We have not lifted any functions out of this paper's repositories yet, so there is nothing we have run. The repositories linked to it are listed below.
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
We study reinforcement learning (RL) with noreward demonstrations, a setting in which an RL agent has access to additional data from the interaction of other agents with the same environment. However, it has no access to the rewards or goals of these agents, and their objectives and levels of expertise may vary widely. These assumptions are common in multi-agent settings, such as autonomous driving. To effectively use this data, we turn to the framework of successor features. This allows us to disentangle shared features and dynamics of the environment from agent-specific rewards and policies. We propose a multi-task inverse reinforcement learning (IRL) algorithm, called inverse temporal difference learning (ITD), that learns shared state features, alongside peragent successor features and preference vectors, purely from demonstrations without reward labels. We further show how to seamlessly integrate ITD with learning from online environment interactions, arriving at a novel algorithm for reinforcement learning with demonstrations, called ΨΦ-learning (pronounced 'Sci-Fi'). We provide empirical evidence for the effectiveness of ΨΦlearning as a method for improving RL, IRL, imitation, and few-shot transfer, and derive worstcase bounds for its performance in zero-shot transfer to new tasks.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2102.12560")
get_code_for_paper("2102.12560")
have("2102.12560")
The run record, dated, one paper per request, free:
curl https://syntology.ai/api/ran/2102.12560.json
A badge for a README (the split and the date, never a ratio):
[](https://syntology.ai/paper/2102.12560)
Connect an agent — have() is free.